22.1: Editable installs for packages with extras & flit causes `UndefinedEnvironmentName` #11110

woodruffw · 2022-05-11T23:36:41Z

Description

First of all, thanks for pip and for taking the time to read this issue!

We use editable installs on sigstore-python to provide a convenient development environment. For example, either with or without the dev extra:

pip install --editable .
pip install --editable .[dev]

Our dependencies are entirely managed within a pyproject.toml, with a small stub setup.py to help pip do an editable install.

Up until pip==22.1, these commands worked just fine. Upon upgrading to 22.1 (released today), we get this error:

  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 215, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.

(I've truncated the traceback. I'll paste the full-length trace below.)

When I run with --debug, I see that the dev extra is being passed in as an environment marker, which then fails since extra is not included in the marker environment.

Expected behavior

I expect pip install --editable . to behave the same as it did in the last stable release (22.0.4), i.e. install correctly with no errors.

pip version

22.1

Python version

3.10.0

OS

macOS 12.3.1

How to Reproduce

These steps should reproduce the failure:

git clone https://github.com/sigstore/sigstore-python && cd sigstore-python
make dev

If that fails to pull down the latest pip for whatever reason, you can do it explicitly:

git clone https://github.com/sigstore/sigstore-python && cd sigstore-python
python -m venv env && source env/bin/activate
pip install --upgrade pip
pip install --editable .

Output

Obtaining file:///Users/william/devel/sigstore-python
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
  Getting requirements to build editable: started
  Getting requirements to build editable: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 340, in run
    requirement_set = resolver.resolve(
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements
    req = self._make_requirement_from_install_req(
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 188, in _make_candidate_from_link
    self._editable_candidate_cache[link] = EditableCandidate(
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 316, in __init__
    super().__init__(
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
    self.dist = self._prepare()
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare
    dist = self._prepare_distribution()
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 326, in _prepare_distribution
    return self._factory.preparer.prepare_editable_requirement(self._ireq)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 543, in prepare_editable_requirement
    dist = _get_prepared_distribution(
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 57, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 45, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 114, in _install_build_reqs
    conflicting, missing = self.req.build_env.check_requirements(build_reqs)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_internal/build_env.py", line 178, in check_requirements
    if req.marker is not None and not req.marker.evaluate():
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 304, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 234, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 215, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.

Code of Conduct

I agree to follow the PSF Code of Conduct.

The text was updated successfully, but these errors were encountered:

pradyunsg · 2022-05-11T23:57:37Z

OK, I can reproduce this by cloning sigstore and running pip install -e ".[dev]" in a clean venv. This is likely related to the fact that we're evaluating markers starting with this release, added in #11030.

This seems to have broken editable installs for any package that has extras and uses setuptools. Also, TIL that setuptools throws all your runtime dependencies as build-time dependencies as well (at least for editable installs). That's... a very interesting choice.

I think the fix for this would be to not have any optional-dependencies installed in the build environment, by passing {"extra": ""} into the marker evaluation. I don't think we should be installing the optional-dependencies into the environment, especially since there's no mechanism to specify what extras should be selected for populating the build environment.

potiuk · 2022-05-12T08:59:25Z

Not sure if this is helpful, but I am just checking Airflow and we have no problems with new pip=22.1 with installing Airflow in editable mode with "[devel]" extras (and we use setuptools too - maybe not in the same way as sigstore-python.

1} Checkout https://github.com/apache/airflow
2) Create & activate virtualenv
3) Upgrade pip to latest
4) ./scripts/tools/initialize_virtualenv.py

One of the things the venv intialization does is:

pip install -e '.[devel]' --constraint https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.9.txt

And it works flawlessly.

sbidoul · 2022-05-12T09:15:02Z

I also checked this scenario on one of my projects and could not reproduce.

Side note: since sigstore-python uses flit, which supports PEP 660 (as of flit_core>=3.4), the setup.py can be removed.
That does not resolve the issue, though.

pradyunsg · 2022-05-12T09:52:38Z

Ah, indeed. I misunderstood that it was using setuptools' experimental support for pyproject.toml, and wasn't looking carefully. :)

This is indeed a flit-specific issue. When it can't figure out what the version of the package is (either by looking at version in pyproject.toml or looking for __version__ = <str> assignments in the top level module via AST), then it installs all the runtime dependencies to be able to import the package during the build: https://github.com/pypa/flit/blob/ec6e6b7c8994d5e79d0fd951d9e4c26a7c5d9977/flit_core/flit_core/buildapi.py#L35

This is happening because sigstore has an additional indirection for loading the version, for some reason, so a simple AST parse is not able to find the version.

So... the use case is significantly less disruptive -- for packages that use flit, define extras and don't want to maintain a static version in pyproject.toml or the module.py/module/__init__.py file.

potiuk · 2022-05-12T10:00:31Z

This possibly could be fixed in sigstore then? It does not seem like worth fixing in pip ?

pfmoore · 2022-05-12T10:33:17Z

This possibly could be fixed in sigstore then? It does not seem like worth fixing in pip ?

There are a number of options here:

sigstore could fix the issue by avoiding the extra indirection.
flit could improve their process for handling the case where they can't find the version statically.
pip could add a workaround as per Filter out build requirements that require an extra to be used #11112

It's worth noting that I agree with @pradyunsg's comment

I don't think we should be installing the optional-dependencies into the environment, especially since there's no mechanism to specify what extras should be selected for populating the build environment.

I think it's worth us adding the workaround, so that our behaviour is explicit, and doesn't trigger an error. But flit needs to decide whether they want to support this usage, given we don't install extras, and possibly fix their approach. And honestly, IMO sigstore should probably just not try to be so clever 🙂 (or to put it another way, I'd be 100% fine with flit choosing to simply error in this case).

flying-sheep · 2022-05-12T11:20:35Z

I’m running into this without editable mode and without specifying extras. There pure existence seems to be enough?

I have a flit project with:

[project.optional-dependencies]
dev = ['boto3-stubs[ecs,s3,sqs,ssm]', ...]

and running pip install . with pip 22.1 gives me

UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.

PS: Way to deflect blame, pip, you’re your own subprocess!

note: This error originates from a subprocess, and is likely not a problem with pip.

pfmoore · 2022-05-12T12:17:29Z

I'm wondering - are we checking the build requirements are installed in every build environment, or just when --no-build-isolation is used? Because it seems to be that if we created the build environment, checking seems to be a bit over-cautious, as we should know we installed the right things?

Regardless, though, checking just non-optional dependencies should avoid the problem. We're not going to install build dependencies that are specified as optional, so a backend (or project) that needs them is probably just broken. But we should ignore them quietly, rather than just fail - being strict here doesn't serve any useful purpose.

flying-sheep · 2022-05-12T12:21:38Z

I would appreciate a quick fix before I have to update all our CI pipelines with pip install -U 'pip!=22.1'

uranusjr · 2022-05-12T12:23:15Z

We currently do check requirements in all build environments. It’s probably a carry-over from the old resolver time, when the build environment can be populated incorrectly.

woodruffw · 2022-05-12T13:45:25Z

And honestly, IMO sigstore should probably just not try to be so clever 🙂 (or to put it another way, I'd be 100% fine with flit choosing to simply error in this case).

I agree completely. I wasn't aware we were triggering such clever behavior! I'm going to make the necessary fixes on sigstore's side, and IMO this should be a hard error in flit as well.

flying-sheep · 2022-05-12T13:58:32Z

Flit supported it so far, so changing it will break a lot of packages, including basically all of mine (private, public, and commercial). Flit should definitely not suddenly stop supporting this.

In the future I’ll use hatch-vcs to make things like this more convenient, but until recently, flit (relying on this behavior) was the best option for me.

BubuOT · 2022-05-12T14:01:03Z

We also ran into this today. We dynamically generate the version via flit-scm so flit getting the version by importing the package is a requirement for us.

pfmoore · 2022-05-12T14:14:20Z

At a minimum, flit could not include optional dependencies when generating the list of build dependencies, as they'll be ignored anyway. But this should be discussed on the flit tracker, not here.

domdfcoding · 2022-05-16T15:43:42Z

I ran into this issue with a normal install from source (i.e. not an editable install). Regardless, #11112 fixed it for me.

uranusjr · 2022-05-16T15:59:21Z

Yeah this is not specific to editable installs, but any PEP 517 scenarios as well. This only happens to editables now because we recently added support to PEP 660, which shares the build environment population logic to PEP 517.

woodruffw added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels May 11, 2022

pradyunsg removed the S: needs triage Issues/PRs that need to be triaged label May 11, 2022

woodruffw mentioned this issue May 11, 2022

bandit: merge config into pyproject sigstore/sigstore-python#86

Merged

pradyunsg changed the title ~~Possible regression: editable install with pyproject.toml fails with 22.1~~ 22.1: Editable installs for packages with extras & setuptools causes UndefinedEnvironmentName May 11, 2022

pradyunsg mentioned this issue May 12, 2022

Filter out build requirements that require an extra to be used #11112

Merged

pradyunsg changed the title ~~22.1: Editable installs for packages with extras & setuptools causes UndefinedEnvironmentName~~ 22.1: Editable installs for packages with extras & flit causes UndefinedEnvironmentName May 12, 2022

woodruffw mentioned this issue May 12, 2022

sigstore: fix build by removing version indirection sigstore/sigstore-python#87

Merged

bollwyvl mentioned this issue May 12, 2022

Editable installs broken with pip 22.1 ipython/traitlets#725

Open

pradyunsg closed this as completed in #11112 May 12, 2022

woodruffw mentioned this issue May 13, 2022

Search more files for __version__? pypa/flit#551

Closed

This was referenced May 15, 2022

Switch to hatch backend ipython/ipyparallel#714

Merged

Use hatch backend jupyter-server/jupyter_server#846

Merged

Switch to hatch build backend jupyter/nbconvert#1777

Merged

Switch to hatch backend ipython/traitlets#728

Merged

pquentin mentioned this issue May 17, 2022

Fix Sphinx links in documentation urllib3/urllib3#2604

Merged

robdimsdale mentioned this issue May 18, 2022

Updates buildpack.toml with new dependency versions: 2022.5.2, 2022.4.30 paketo-buildpacks/pipenv#206

Closed

dummyindex mentioned this issue May 19, 2022

fix build workflow due to pip requiring extras in its latest version aristoteleo/dynamo-release#357

Merged

This was referenced May 20, 2022

pip 22.1 failing when installing pyparsing from source; fails on 'extra' #11132

Closed

v22 breaks flit optional-dependencies #11133

Closed

branchvincent mentioned this issue May 22, 2022

python@3.9: update resources Homebrew/homebrew-core#102092

Closed

6 tasks

hramezani mentioned this issue May 23, 2022

Unpin pip in docs/requirements.txt urllib3/urllib3#2611

Merged

github-actions bot locked as resolved and limited conversation to collaborators Jun 17, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

22.1: Editable installs for packages with extras & flit causes `UndefinedEnvironmentName` #11110

22.1: Editable installs for packages with extras & flit causes `UndefinedEnvironmentName` #11110

woodruffw commented May 11, 2022

pradyunsg commented May 11, 2022 •

edited

potiuk commented May 12, 2022

sbidoul commented May 12, 2022

pradyunsg commented May 12, 2022 •

edited

potiuk commented May 12, 2022

pfmoore commented May 12, 2022

flying-sheep commented May 12, 2022 •

edited

pfmoore commented May 12, 2022

flying-sheep commented May 12, 2022

uranusjr commented May 12, 2022

woodruffw commented May 12, 2022

flying-sheep commented May 12, 2022 •

edited

BubuOT commented May 12, 2022

pfmoore commented May 12, 2022

domdfcoding commented May 16, 2022

uranusjr commented May 16, 2022

22.1: Editable installs for packages with extras & flit causes UndefinedEnvironmentName #11110

22.1: Editable installs for packages with extras & flit causes UndefinedEnvironmentName #11110

Comments

woodruffw commented May 11, 2022

Description

Expected behavior

pip version

Python version

OS

How to Reproduce

Output

Code of Conduct

pradyunsg commented May 11, 2022 • edited

potiuk commented May 12, 2022

sbidoul commented May 12, 2022

pradyunsg commented May 12, 2022 • edited

potiuk commented May 12, 2022

pfmoore commented May 12, 2022

flying-sheep commented May 12, 2022 • edited

pfmoore commented May 12, 2022

flying-sheep commented May 12, 2022

uranusjr commented May 12, 2022

woodruffw commented May 12, 2022

flying-sheep commented May 12, 2022 • edited

BubuOT commented May 12, 2022

pfmoore commented May 12, 2022

domdfcoding commented May 16, 2022

uranusjr commented May 16, 2022

22.1: Editable installs for packages with extras & flit causes `UndefinedEnvironmentName` #11110

22.1: Editable installs for packages with extras & flit causes `UndefinedEnvironmentName` #11110

pradyunsg commented May 11, 2022 •

edited

pradyunsg commented May 12, 2022 •

edited

flying-sheep commented May 12, 2022 •

edited

flying-sheep commented May 12, 2022 •

edited