Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@ git+ requirements with backtracking resolver can result in pins to local wheel cache #1647

Closed
gschaffner opened this issue Jun 30, 2022 · 1 comment · Fixed by #1649
Closed
Labels
bug Something is not working resolver Related to dependency resolver
Milestone

Comments

@gschaffner
Copy link
Member

gschaffner commented Jun 30, 2022

If an @ git+ requirement is present in pip's wheel cache, pip-compile --resolver backtracking will pin to @ file://....whl.

(Moved from #1539 (comment). @ git+ pinned package switched from pip-tools to typeguard for clarity.)

Environment Versions

  1. OS Type: Manjaro 21.3.1
  2. Python version: CPython 3.10.5
  3. pip version: 22.1.2
  4. pip-tools version: 1e05d00

Steps to replicate

#!/usr/bin/env bash

set -Eeuo pipefail
shopt -s inherit_errexit

cd "$(mktemp -d)"

echo 'typeguard @ git+https://github.com/agronholm/typeguard.git@a298d6cde249fe7ad46b90b94aa3bdd246e60417' > requirements.in
# cache ~/.cache/pip/wheels/a3/ac/4a/1aa2eba1bf7a5f0259fbc2f4160621fe8ea72613653e32c6fe/typeguard-2.13.3.post48-py3-none-any.whl
python -m venv --upgrade-deps .venv && . .venv/bin/activate
mv ~/.cache/pip/wheels{,.bak}
pip install typeguard==2.13.3
pip install -r requirements.in
# pip-compile
rm -r .venv
python -m venv --upgrade-deps .venv && . .venv/bin/activate
pip install 'pip-tools @ git+https://github.com/jazzband/pip-tools.git@1e05d0028ce16d4b2de53ae38a33df7feb84e800'
pip-compile --resolver backtracking
# restore cache
rm -r ~/.cache/pip/wheels
mv ~/.cache/pip/wheels{.bak,}

Expected result

typeguard @ git+https://github.com/agronholm/typeguard.git@a298d6cde249fe7ad46b90b94aa3bdd246e60417

Actual result

< ... truncated for brevity ... >
typeguard @ file:///home/ganden/.cache/pip/wheels/a3/ac/4a/1aa2eba1bf7a5f0259fbc2f4160621fe8ea72613653e32c6fe/typeguard-2.13.3.post48-py3-none-any.whl
    # via -r requirements.in
@atugushev atugushev added the bug Something is not working label Jun 30, 2022
@atugushev
Copy link
Member

atugushev commented Jun 30, 2022

At the first glance these might be the fixes:

  1. Override cached link by source link
diff --git piptools/resolver.py piptools/resolver.py
index f9d3d12..72c95d5 100644
--- piptools/resolver.py
+++ piptools/resolver.py
@@ -763,7 +763,10 @@ def _get_install_requirement_from_candidate(

         # Prepare pinned install requirement. Copy it from candidate's install
         # requirement so that it could be mutated later.
-        pinned_ireq = copy_install_requirement(ireq)
+        kwargs = {}
+        if candidate.source_link is not None:
+            kwargs['link'] = candidate.source_link
+        pinned_ireq = copy_install_requirement(ireq, **kwargs)

         # Canonicalize name
         assert ireq.name is not None
  1. Hack writer
diff --git piptools/utils.py piptools/utils.py
index 47f3637..c7624dd 100644
--- piptools/utils.py
+++ piptools/utils.py
@@ -119,7 +119,9 @@ def format_requirement(
     in a less verbose way than using its `__str__` method.
     """
     if ireq.editable:
-        line = f"-e {ireq.link.url}"
+        req = ireq.req
+        url = req.url if req and req.url else ireq.link.url
+        line = f"-e {url}"
     elif is_url_requirement(ireq):
         line = _build_direct_reference_best_efforts(ireq)
     else:

However, need some research on how InstallRequirement.link and caches work.

@atugushev atugushev added the resolver Related to dependency resolver label Jun 30, 2022
@AndydeCleyre AndydeCleyre added this to the 6.8.1 milestone Jul 25, 2022
atugushev pushed a commit that referenced this issue Nov 3, 2022
For Windows support, this includes workarounds for
python/cpython#85815.
dand-oss pushed a commit to dand-oss/pip-tools that referenced this issue Nov 12, 2022
For Windows support, this includes workarounds for
python/cpython#85815.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is not working resolver Related to dependency resolver
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants