Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unstable (non-idempotent) isort behavior. #1546

Closed
apollo13 opened this issue Oct 8, 2020 · 4 comments · Fixed by hypothesis/viahtml#107 or wwade/jobrunner#63
Closed

Unstable (non-idempotent) isort behavior. #1546

apollo13 opened this issue Oct 8, 2020 · 4 comments · Fixed by hypothesis/viahtml#107 or wwade/jobrunner#63
Labels
bug Something isn't working

Comments

@apollo13
Copy link

apollo13 commented Oct 8, 2020

I am seeing something weird on isort 5.6.0 and didn't seem to find something similar in the issue. When running isort tests src against my project (https://github.com/rocketDuck/dobby) multiple times I get the following output:

(.venv) ➜  dobby git:(master) ✗ isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py
(.venv) ➜  dobby git:(master) ✗ isort tests src
(.venv) ➜  dobby git:(master) ✗ isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
(.venv) ➜  dobby git:(master) isort tests src
Fixing /home/florian/dev/dobby/tests/test_templating.py

As you can see isort fixes the file multiple times (adding and removing a blank line between imports) without any changes by me. I literally ran those commands one after another.

Additional information:

python --version
Python 3.8.6

Running with --verbose shows that isort sometimes considers dobby FIRST and sometimes THIRDPARTY:

from-type place_module for pathlib returned STDLIB
else-type place_module for pytest returned THIRDPARTY
from-type place_module for dobby.templates returned THIRDPARTY
else-type place_module for click returned THIRDPARTY
else-type place_module for inspect returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for functools returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for httpx returned THIRDPARTY
from-type place_module for jinja2.exceptions returned THIRDPARTY
from-type place_module for .config returned LOCALFOLDER
else-type place_module for os returned STDLIB
else-type place_module for sys returned STDLIB
else-type place_module for time returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER
else-type place_module for functools returned STDLIB
else-type place_module for json returned STDLIB
else-type place_module for os returned STDLIB
else-type place_module for pathlib returned STDLIB
else-type place_module for re returned STDLIB
else-type place_module for yaml returned THIRDPARTY
from-type place_module for dotenv.main returned THIRDPARTY
from-type place_module for jinja2 returned THIRDPARTY
from-type place_module for jinja2.runtime returned THIRDPARTY
from-type place_module for dobby.cli returned THIRDPARTY
else-type place_module for ssl returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for pathlib returned STDLIB
from-type place_module for urllib.parse returned STDLIB
else-type place_module for click returned THIRDPARTY
else-type place_module for httpx returned THIRDPARTY
from-type place_module for httpx._config returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER

vs

from-type place_module for pathlib returned STDLIB
else-type place_module for pytest returned THIRDPARTY
from-type place_module for dobby.templates returned FIRSTPARTY
Fixing /home/florian/dev/dobby/tests/test_templating.py
else-type place_module for click returned THIRDPARTY
else-type place_module for inspect returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for functools returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for httpx returned THIRDPARTY
from-type place_module for jinja2.exceptions returned THIRDPARTY
from-type place_module for .config returned LOCALFOLDER
else-type place_module for os returned STDLIB
else-type place_module for sys returned STDLIB
else-type place_module for time returned STDLIB
else-type place_module for click returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER
else-type place_module for functools returned STDLIB
else-type place_module for json returned STDLIB
else-type place_module for os returned STDLIB
else-type place_module for pathlib returned STDLIB
else-type place_module for re returned STDLIB
else-type place_module for yaml returned THIRDPARTY
from-type place_module for dotenv.main returned THIRDPARTY
from-type place_module for jinja2 returned THIRDPARTY
from-type place_module for jinja2.runtime returned THIRDPARTY
from-type place_module for dobby.cli returned FIRSTPARTY
else-type place_module for ssl returned STDLIB
else-type place_module for sys returned STDLIB
from-type place_module for pathlib returned STDLIB
from-type place_module for urllib.parse returned STDLIB
else-type place_module for click returned THIRDPARTY
else-type place_module for httpx returned THIRDPARTY
from-type place_module for httpx._config returned THIRDPARTY
from-type place_module for . returned LOCALFOLDER

As to why isort behaves differently on subsequent runs I have no idea.

@apollo13
Copy link
Author

apollo13 commented Oct 8, 2020

Isort itself is installed in the current active environment. I am using poetry to manage dependencies. If you clone my project this should be able to reproduce it:

poetry install
poetry shell
isort tests src
isort tests src
isort tests src

@apollo13
Copy link
Author

apollo13 commented Oct 8, 2020

I left my repository at the current broken state. I know that I can use known_first_party and similar config options, but it would be nice if isort were stable either way.

@timothycrosley timothycrosley added the bug Something isn't working label Oct 8, 2020
@timothycrosley
Copy link
Member

Thank you for reporting! This issue has been resolved in a hotfix release (5.6.1) and your project has been added to isort's integration test suite.

Any time non-idempotent behaviour is observed with isort it is of major concern for the project. I was able to pinpoint the cause of the issue to two things:

  1. The just released support for automatic detection of namespace packages misidentifying for repos with no python files in root. This caused the mis-categorization.
  2. The use of frozenset (which has non-deterministic ordering) internally to de-dup source paths. This caused the non-idempotent behaviour, and has been an undiscovered bug for some time.

Both of these issues have been resolved.

Thanks!

~Timothy

@apollo13
Copy link
Author

apollo13 commented Oct 8, 2020

Thank you for the quick turnaround and explanation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants