Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No such file or directory error when installing types with mypy #16767

Open
jakecyr opened this issue Jan 10, 2024 · 5 comments · May be fixed by #16772
Open

No such file or directory error when installing types with mypy #16767

jakecyr opened this issue Jan 10, 2024 · 5 comments · May be fixed by #16772
Labels
bug mypy got something wrong

Comments

@jakecyr
Copy link

jakecyr commented Jan 10, 2024

Bug Report

When running the command poetry run mypy --config-file pyproject.toml in GitLab CI, MyPy starts installing types as expected, but then throws an error:

mypy: can't read file '/root/.cache/pypoetry/virtualenvs/our-package-d4xTn211-py3.10/lib/python3.10/site-packages//google': No such file or directory

Expected Behavior

I would expect MyPy to install the required types and run the static analysis check.

Actual Behavior

Installing collected packages: types-PyYAML, types-pytz, types-PyMySQL, urllib3, types-protobuf, pycparser, types-requests, cffi, cryptography, types-pyOpenSSL, types-redis
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.26.18
    Uninstalling urllib3-1.26.18:
      Successfully uninstalled urllib3-1.26.18
Successfully installed cffi-1.16.0 cryptography-41.0.7 pycparser-2.21 types-PyMySQL-1.1.0.1 types-PyYAML-6.0.12.12 types-protobuf-4.24.0.20240106 types-pyOpenSSL-23.3.0.20240106 types-pytz-2023.3.1.1 types-redis-4.6.0.20240106 types-requests-2.31.0.20240106 urllib3-2.1.0
[notice] A new release of pip is available: 23.3.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
mypy: can't read file '/root/.cache/pypoetry/virtualenvs/our-package-d4xTn211-py3.10/lib/python3.10/site-packages//google': No such file or directory
Installing missing stub packages:
/root/.cache/pypoetry/virtualenvs/our-package-d4xTn211-py3.10/bin/python -m pip install types-PyMySQL types-PyYAML types-protobuf types-pytz types-redis types-requests

Your Environment

This error occurs when running the mypy command in GitLab CI.

  • Mypy version used: 1.8.0 (also tried with 1.7.0)
  • Mypy command-line flags: --config-file pyproject.toml
  • Mypy configuration options from mypy.ini (and other config files): 
    [tool.mypy]
disallow_untyped_defs = true
allow_any_unimported = true
no_implicit_optional = true
check_untyped_defs = true
warn_return_any = true
warn_unused_ignores = true
show_error_codes = true
warn_unused_configs = true
disallow_incomplete_defs = true
ignore_missing_imports = true
files = ["our_package", "examples"]
exclude = [".venv", ".cache", "build"]
install_types = false
non_interactive = true
pretty = true
  • Python version used: 3.10
@jakecyr jakecyr added the bug mypy got something wrong label Jan 10, 2024
@jakecyr
Copy link
Author

jakecyr commented Jan 10, 2024 •
edited

It seemed that my previous pipeline jobs that were succeeding started failing so I look at the versions of the typing packages being installed and noticed these were updated since (working versions are below):

types-protobuf = "4.24.0.4"
types-pyopenssl = "23.3.0.0"
types-redis = "4.6.0.11"

Locking the versions to the versions listed above fixed the error. I'm not sure what the issue is with the new typing packages, but it seems either mypy is pulling in incompatible versions or the there is a bug in latest version of one of the packages.

@odimko
Copy link

odimko commented Jan 11, 2024

@jakecyr thanks for your comment! it helped a lot.

waiting until this is fixed now python/typeshed#11254

@getim
Copy link
Contributor

getim commented Jan 11, 2024 •
edited

We ran into this as well and I drilled down into mypy since yesterday:

  • It looks like the logic here assumes that there's more than 1 component in the import path being resolved. However, when that's not the case just (e.g. just google), dir_chain is empty. From there runtime_path calculated here contains a trailing slash.
  • Later on, this path gets used as the prefix in exists_case. The trailing slash breaks the startswith check because something like foo doesn't start with foo/. That means exists_case ends up returning True for something like site-packages//google while it doesn't exist.

I'm just laying this out here because I'm now unsure which component is wrong exactly. The runtime_path assignment can check if dir_chain is empty? Should exists_case be fixed to correctly handle the trailing slash? Both seem to fix the issue from my tests.

Edit: I guess my comment is more about the fact that mypy could handle this better, even if the linked typeshed issue gets fixed.

@hauntsaninja
Copy link
Collaborator

@getim thanks for looking into it! While typeshed will need to be fixed, let's remove trailing slash in exists_case — seems less of a sharp edge and matches os.path.exists better. Interested in opening a PR?

@getim getim linked a pull request Jan 12, 2024 that will close this issue
Open
@getim
Copy link
Contributor

getim commented Jan 12, 2024

@hauntsaninja Sounds good, see here

@Hackerman342 Hackerman342 mentioned this issue Jan 16, 2024
Merged
@tianweidut tianweidut mentioned this issue Jan 17, 2024
Draft
5 tasks
szysad pushed a commit to neptune-ai/neptune-client that referenced this issue Jan 18, 2024
add neptune backend protobuf calls for series values fetching.
299000b 
Additionally:
  - add unit test for new call. Includes some refractoring as new calls are ment to be new implementation of already exisiting calls.
  - add protobuf python compliled code with stubs in neptune.internal.proto with additional unit tests
  - pin types-protobuf in mypy pre-commit hook as a workaround to fresh protobuf-typing bug. More info: python/mypy#16767
  - excluded proto files from flake8 check as they are output of protoc compiler and far from perfect syntax
@szysad szysad mentioned this issue Jan 18, 2024
Closed
2 tasks
szysad pushed a commit to neptune-ai/neptune-client that referenced this issue Jan 22, 2024
add neptune backend protobuf calls for series values fetching.
6d68e43 
Additionally:
  - add unit test for new call. Includes some refractoring as new calls are ment to be new implementation of already exisiting calls.
  - add protobuf python compliled code with stubs in neptune.internal.proto with additional unit tests
  - pin types-protobuf in mypy pre-commit hook as a workaround to fresh protobuf-typing bug. More info: python/mypy#16767
  - excluded proto files from flake8 check as they are output of protoc compiler and far from perfect syntax
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug mypy got something wrong
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fscache: handle trailing slash in exists_case prefix getim/mypy
4 participants
@getim @hauntsaninja @jakecyr @odimko