Skip to content

Commit

Permalink
improve is_namespace check
Browse files Browse the repository at this point in the history
See https://stackoverflow.com/a/42962529.

Let's take the following contents as an example:
```python
import celery.result

```

From pylint-dev#1777, astroid started to use `processed_components` for
namespace check. In the above case, the `modname` is `celery.result`,
it first checks for `celery` and then `celery.result`.
Before that PR, it'd always check for `celery.result`.

`celery` is recreating module to make it lazily load.
See
https://github.com/celery/celery/blob/34533ab44d2a6492004bc3df44dc04ad5c6611e7/celery/__init__.py#L150.

This module does not have `__spec__` set.

Reading through Python's docs, it seems that `__spec__` can be set
to None, so it seems like it's not a thing that we can depend upon
for namespace checks.

See https://docs.python.org/3/reference/import.html#spec__.

---

The `celery.result` gets imported for me when pylint-pytest plugin tries
to load fixtures, but this could happen anytime if any plugin imports
packages. In that case, `importlib.util._find_spec_from_path("celery")` will raise ValueError
since it's already in `sys.modules` and does not have a spec.

Fixes pylint-dev/pylint#7488.
  • Loading branch information
skshetry committed Sep 21, 2022
1 parent 849d043 commit 0d9e05c
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 1 deletion.
4 changes: 4 additions & 0 deletions ChangeLog
Expand Up @@ -12,6 +12,10 @@ Release date: TBA

Refs PyCQA/pylint#5151

* Improve detection of namespace packages for the modules with ``__spec__`` set to None.

Closes PyCQA/pylint#7488.


What's New in astroid 2.12.11?
==============================
Expand Down
5 changes: 4 additions & 1 deletion astroid/interpreter/_import/util.py
Expand Up @@ -52,8 +52,11 @@ def is_namespace(modname: str) -> bool:
# Check first fragment of modname, e.g. "astroid", not "astroid.interpreter"
# because of cffi's behavior
# See: https://github.com/PyCQA/astroid/issues/1776
mod = sys.modules[processed_components[0]]
return (
sys.modules[processed_components[0]].__spec__ is None
mod.__spec__ is None
and getattr(mod, "__file__", None) is None
and hasattr(mod, "__path__")
and not IS_PYPY
)
except KeyError:
Expand Down
9 changes: 9 additions & 0 deletions tests/unittest_manager.py
Expand Up @@ -144,6 +144,15 @@ def test_module_unexpectedly_missing_spec(self) -> None:
finally:
astroid_module.__spec__ = original_spec

def test_module_unexpectedly_spec_is_none(self) -> None:
astroid_module = sys.modules["astroid"]
original_spec = astroid_module.__spec__
astroid_module.__spec__ = None
try:
self.assertFalse(util.is_namespace("astroid"))
finally:
astroid_module.__spec__ = original_spec

def test_implicit_namespace_package(self) -> None:
data_dir = os.path.dirname(resources.find("data/namespace_pep_420"))
contribute = os.path.join(data_dir, "contribute_to_namespace")
Expand Down

0 comments on commit 0d9e05c

Please sign in to comment.