Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable importlib.metadata backend on Python 3.11 #11044

Merged
merged 1 commit into from
May 10, 2022
Merged

Enable importlib.metadata backend on Python 3.11 #11044

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,11 +219,12 @@ jobs:
env:
TEMP: "R:\\Temp"

# TODO: Remove this when we add Python 3.11 to CI.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do this now FWIW, 3.11.0-alpha - 3.11?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it’s a bit difficult to convert 3.11.0-alpha.7 to 3.11 for nox -s test-{{ python }}…

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use the include field to add an additional key-value to the 3.11.0-alpha.7 entry -> https://github.com/cooperlees/black-primer/blob/6c40328df33f72468105466423edd2115be491ff/.github/workflows/ci.yaml#L30-L35 and then use a OR expression -> https://github.com/cooperlees/black-primer/blob/6c40328df33f72468105466423edd2115be491ff/.github/workflows/ci.yaml#L53

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the syntax I was looking for, thanks

${{ matrix.nox-python || matrix.python }}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seem like it’s not ready for virtualenv (which we need for Nox) unfortunately

ERROR: The executable /home/runner/work/pip/pip/.nox/test-3-11/bin/python3.11 is not functioning
ERROR: It thinks sys.prefix is '/opt/hostedtoolcache/Python/3.11.0-alpha.7/x64' (should be '/home/runner/work/pip/pip/.nox/test-3-11')
ERROR: virtualenv is not compatible with this system or executable
Running virtualenv with interpreter /opt/hostedtoolcache/Python/3.11.0-alpha.7/x64/bin/python3.11

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we're stuck on virtualenv 16.x for our test suite. That's now a proper problem. :)

tests-importlib-metadata:
name: tests for importlib.metadata backend
runs-on: ubuntu-latest
env:
_PIP_METADATA_BACKEND_IMPORTLIB: egg-compat
_PIP_USE_IMPORTLIB_METADATA: 'true'

needs: [pre-commit, packaging, determine-changes]
if: >-
Expand All @@ -241,7 +242,6 @@ jobs:

- run: pip install nox 'virtualenv<20'

# Main check
- name: Run unit tests
run: >-
nox -s test-3.10 --
Expand Down
4 changes: 4 additions & 0 deletions news/11044.process.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Enable the ``importlib.metadata`` metadata implementation by default on
Python 3.11 (or later). The environment variable ``_PIP_USE_IMPORTLIB_METADATA``
can still be used to enable the implementation on 3.10 and earlier, or disable
it on 3.11 (by setting it to ``0`` or ``false``).
29 changes: 28 additions & 1 deletion src/pip/_internal/metadata/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import contextlib
import functools
import os
import sys
from typing import TYPE_CHECKING, List, Optional, Type, cast

from pip._internal.utils.misc import strtobool

from .base import BaseDistribution, BaseEnvironment, FilesystemWheel, MemoryWheel, Wheel

if TYPE_CHECKING:
Expand All @@ -22,14 +26,37 @@
]


def _should_use_importlib_metadata() -> bool:
"""Whether to use the ``importlib.metadata`` or ``pkg_resources`` backend.

By default, pip uses ``importlib.metadata`` on Python 3.11+, and
``pkg_resourcess`` otherwise. This can be overriden by a couple of ways:

* If environment variable ``_PIP_USE_IMPORTLIB_METADATA`` is set, it
dictates whether ``importlib.metadata`` is used, regardless of Python
version.
* On Python 3.11+, Python distributors can patch ``importlib.metadata``
to add a global constant ``_PIP_USE_IMPORTLIB_METADATA = False``. This
makes pip use ``pkg_resources`` (unless the user set the aforementioned
environment variable to *True*).
"""
with contextlib.suppress(KeyError, ValueError):
return bool(strtobool(os.environ["_PIP_USE_IMPORTLIB_METADATA"]))
if sys.version_info < (3, 11):
return False
import importlib.metadata

return bool(getattr(importlib.metadata, "_PIP_USE_IMPORTLIB_METADATA", True))


class Backend(Protocol):
Distribution: Type[BaseDistribution]
Environment: Type[BaseEnvironment]


@functools.lru_cache(maxsize=None)
def select_backend() -> Backend:
if os.environ.get("_PIP_METADATA_BACKEND_IMPORTLIB"):
if _should_use_importlib_metadata():
from . import importlib

return cast(Backend, importlib)
Expand Down