pip-compile solving failure with tensorflow when solution exists

[jli]

pip-compile fails to solve my requirements.in when a solution exists and pip install works.

(This seems quite similar to #1420. I'm not sure if it's the same underlying bug, so just filing a new issue.)

Environment Versions

1. OS Type: docker image: amd64/python:3.8-buster
2. Python version: 3.8.10
3. pip version: 21.1.2
4. pip-tools version: 6.2.0

Steps to replicate

$ docker run --rm -it --entrypoint bash amd64/python:3.8-buster
$ pip install -U pip wheel pip-tools
$ cat > requirements.in << EOF
tensorflow>=2.4.0
tensorflow-transform==0.29.0
EOF
$ pip-compile

Expected result

The following is a valid solution (not a full result, just listing the packages mentioned in the error message below):

tensorflow==2.4.2
tensorflow-serving-api==2.4.1
tensorflow-transform==0.29.0
tfx-bsl==0.29.0

Actual result

Could not find a version that matches tensorflow!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,<2.5,<3,>=1.15.2,>=2.4.0,>=2.5.0 (from -r requirements.in (line 1))
Tried: 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.4.0, 2.4.1, 2.4.2, 2.5.0
Skipped pre-versions: 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.4.0rc0, 2.4.0rc1, 2.4.0rc2, 2.4.0rc3, 2.4.0rc4, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3
There are incompatible versions in the resolved dependencies:
  tensorflow>=2.4.0 (from -r requirements.in (line 1))
  tensorflow!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,<2.5,>=1.15.2 (from tensorflow-transform==0.29.0->-r requirements.in (line 2))
  tensorflow!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,<3,>=1.15.2 (from tfx-bsl==0.29.0->tensorflow-transform==0.29.0->-r requirements.in (line 2))
  tensorflow<3,>=2.5.0 (from tensorflow-serving-api==2.5.1->tfx-bsl==0.29.0->tensorflow-transform==0.29.0->-r requirements.in (line 2))

Other notes

• I'm not sure why the final line in the error says that tensorflow-serving-api==2.5.1 is required by tfx-bsl==0.29.0. From the tfx-bsl setup.py, you can see that tensorflow-server-api 2.4.* is allowed: https://github.com/tensorflow/tfx-bsl/blob/1c2d94198759642627b9d64f631adaf4af0968fd/setup.py#L176
• Adding tensorflow-serving-api==2.4.* to the requirements.in as a "hint" allows pip-compile run successfully.

[webknjaz]

pip-compile fails to solve my requirements.in when a solution exists and pip install works.

You could verify if choosing to use an old resolver with pip install fails to see if it's related to pip-tools still not using the new one.

[jli]

pip-compile fails to solve my requirements.in when a solution exists and pip install works.

You could verify if choosing to use an old resolver with pip install fails to see if it's related to pip-tools still not using the new one.

I tried to use both the new and the old/deprecated resolver, and both worked and succeeded in installing the requirements file which pip-compile fails on.

Here's what I did:

1. A recent pip version, with new resolver:

$ docker run --rm -it --entrypoint bash amd64/python:3.8-buster
# cat > requirements.in << EOF
tensorflow>=2.4.0
tensorflow-transform==0.29.0
EOF
# python -m venv /venv-pip-plain
# source /venv-pip-plain/bin/activate
# pip -V
pip 21.1.1 from /venv-pip-plain/lib/python3.8/site-packages/pip (python 3.8)
# pip install -r requirements.in
# pip freeze | grep -E '(tensorflow|tfx)'
tensorflow==2.4.2
tensorflow-estimator==2.4.0
tensorflow-metadata==0.29.0
tensorflow-serving-api==2.4.1
tensorflow-transform==0.29.0
tfx-bsl==0.29.0

2. Try to use legacy resolver with --use-deprecated=legacy-resolver flag:

# deactivate
# python -m venv /venv-pip-legacy
# source /venv-pip-legacy/bin/activate
# pip install --use-deprecated=legacy-resolver -r requirements.in
# pip freeze | grep -E '(tensorflow|tfx)'
tensorflow==2.5.0
tensorflow-estimator==2.5.0
tensorflow-metadata==0.29.0
tensorflow-serving-api==2.5.1
tensorflow-transform==0.29.0
tfx-bsl==0.29.0

3. Since it seems like maybe the legacy resolver was removed in pip 21.0, I also tried pip 20.2, which was before the new resolver was added:

# deactivate
# python -m venv /venv-pip-old
# source /venv-pip-old/bin/activate
# pip install pip==20.2.*
# pip -V
pip 20.2.4 from /venv-pip-old/lib/python3.8/site-packages/pip (python 3.8)
# pip install -r requirements.in
# pip freeze | grep -E '(tensorflow|tfx)'
tensorflow==2.5.0
tensorflow-estimator==2.5.0
tensorflow-metadata==0.29.0
tensorflow-serving-api==2.5.1
tensorflow-transform==0.29.0
tfx-bsl==0.29.0

[jli]

In case it's useful, here's the full pip-compile -v output: https://gist.github.com/jli/fd656ab3027c8fc8ee7ed2702a1df5a2

(edit: moved giant wall of text to a gist)

[sawatsky]

pip-tools==6.4.0 is having the same issue with:

numpy==1.21.4
shap==0.37.0

And getting:

There are incompatible versions in the resolved dependencies:
  numpy==1.21.4 (from -r requirements.txt (line 1))
  numpy>=1.14.6 (from scikit-learn==1.0.1->shap==0.37.0->-r requirements.txt (line 2))
  numpy<1.23.0,>=1.16.5 (from scipy==1.7.2->shap==0.37.0->-r requirements.txt (line 2))
  numpy (from shap==0.37.0->-r requirements.txt (line 2))
  numpy>=1.17.3 (from pandas==1.3.4->shap==0.37.0->-r requirements.txt (line 2))
  numpy<1.21,>=1.17 (from numba==0.54.1->shap==0.37.0->-r requirements.txt (line 2))

But pip==21.3.1 installs them successfully.

[jli]

In case it's helpful for others, I suggested an alternative approach to using pip-compile in this comment on a sister bug: #1420 (comment)

[vikingphoenix]

using pip-compile (6.4.0) with requirements.in 1. ccxt, and 2. celery:

There are incompatible versions in the resolved dependencies:
  setuptools<59.7.0,>=59.1.1 (from celery==5.2.3->-r requirements.in (line 2))
  setuptools>=60.9.0 (from ccxt==1.73.65->-r requirements.in (line 1))

perhaps the 2nd version requirement is being skipped from celery?

pip install -r requirements (21.3.1) in works just fine

[atugushev]

It should work with the upcoming backtracking resolver (see #1539).

Click to expand results

$ pip-compile --resolver backtracking
Using indexes:
  https://pypi.org/simple

#
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
#    pip-compile --resolver=backtracking
#
aiobotocore==2.1.1
    # via s3fs
aiohttp==3.8.1
    # via
    #   aiobotocore
    #   aiohttp-retry
    #   fsspec
    #   s3fs
aiohttp-retry==2.4.6
    # via dvc
aioitertools==0.9.0
    # via aiobotocore
aiosignal==1.2.0
    # via aiohttp
appdirs==1.4.4
    # via dvc
async-timeout==4.0.2
    # via aiohttp
asyncssh==2.8.1
    # via scmrepo
atpublic==3.0.1
    # via flufl-lock
attrs==21.4.0
    # via aiohttp
boto3==1.20.24
    # via
    #   -r requirements.in
    #   aiobotocore
botocore==1.23.24
    # via
    #   aiobotocore
    #   boto3
    #   s3transfer
certifi==2021.10.8
    # via
    #   dulwich
    #   requests
cffi==1.15.0
    # via
    #   cryptography
    #   pygit2
charset-normalizer==2.0.12
    # via
    #   aiohttp
    #   requests
colorama==0.4.4
    # via
    #   dvc
    #   rich
commonmark==0.9.1
    # via rich
configobj==5.0.6
    # via dvc
cryptography==36.0.1
    # via asyncssh
dictdiffer==0.9.0
    # via dvc
diskcache==5.4.0
    # via dvc
distro==1.7.0
    # via dvc
dpath==2.0.6
    # via dvc
dulwich==0.20.32
    # via scmrepo
dvc[s3]==2.9.4
    # via -r requirements.in
flatten-dict==0.4.2
    # via dvc
flufl-lock==7.0
    # via dvc
frozenlist==1.3.0
    # via
    #   aiohttp
    #   aiosignal
fsspec[http]==2022.1.0
    # via
    #   dvc
    #   s3fs
    #   scmrepo
ftfy==6.1.1
    # via python-benedict
funcy==1.17
    # via
    #   dvc
    #   scmrepo
future==0.18.2
    # via grandalf
gitdb==4.0.9
    # via gitpython
gitpython==3.1.27
    # via scmrepo
grandalf==0.6
    # via dvc
idna==3.3
    # via
    #   requests
    #   yarl
importlib-resources==5.4.0
    # via dvc
jmespath==0.10.0
    # via
    #   boto3
    #   botocore
mailchecker==4.1.12
    # via python-benedict
multidict==6.0.2
    # via
    #   aiohttp
    #   yarl
nanotime==0.5.2
    # via dvc
networkx==2.6.3
    # via dvc
packaging==21.3
    # via dvc
pathspec==0.9.0
    # via
    #   dvc
    #   scmrepo
phonenumbers==8.12.43
    # via python-benedict
ply==3.11
    # via dvc
psutil==5.9.0
    # via
    #   dvc
    #   flufl-lock
pyasn1==0.4.8
    # via dvc
pycparser==2.21
    # via cffi
pydot==1.4.2
    # via dvc
pygit2==1.8.0
    # via scmrepo
pygments==2.11.2
    # via rich
pygtrie==2.4.2
    # via
    #   dvc
    #   scmrepo
pyparsing==3.0.7
    # via
    #   dvc
    #   grandalf
    #   packaging
    #   pydot
python-benedict==0.25.0
    # via dvc
python-dateutil==2.8.2
    # via
    #   botocore
    #   python-benedict
python-fsutil==0.6.0
    # via python-benedict
python-slugify==6.0.1
    # via python-benedict
pyyaml==6.0
    # via python-benedict
requests==2.27.1
    # via
    #   dvc
    #   fsspec
    #   python-benedict
rich==11.2.0
    # via dvc
ruamel-yaml==0.17.21
    # via dvc
ruamel-yaml-clib==0.2.6
    # via ruamel-yaml
s3fs[boto3]==2022.1.0
    # via dvc
s3transfer==0.5.1
    # via boto3
scmrepo==0.0.7
    # via dvc
shortuuid==1.0.8
    # via dvc
shtab==1.5.3
    # via dvc
six==1.16.0
    # via
    #   configobj
    #   flatten-dict
    #   python-dateutil
smmap==5.0.0
    # via gitdb
tabulate==0.8.9
    # via dvc
text-unidecode==1.3
    # via python-slugify
toml==0.10.2
    # via
    #   dvc
    #   python-benedict
tqdm==4.62.3
    # via dvc
typing-extensions==4.1.1
    # via
    #   aioitertools
    #   dvc
urllib3==1.26.8
    # via
    #   botocore
    #   dulwich
    #   requests
voluptuous==0.12.2
    # via dvc
wcwidth==0.2.5
    # via ftfy
wrapt==1.13.3
    # via aiobotocore
xmltodict==0.12.0
    # via python-benedict
yarl==1.7.2
    # via aiohttp
zc-lockfile==2.0
    # via dvc
zipp==3.7.0
    # via importlib-resources

# The following packages are considered to be unsafe in a requirements file:
# setuptools

[shaib]

I think I have a similar example, the backtracking resolver does not solve it. Pip does.

$ cat requirements.in 
google-api-python-client==1.7.11
google-auth-oauthlib==0.4.2
google-auth-httplib2==0.0.3
firebase-admin==4.5.0
aiogoogle==1.1.13
google-auth==1.20.1

Latest release (6.5.1) does essentially what OP compained about: Pins second-order dependencies too high, too early:

$ pip-compile 
Could not find a version that matches google-auth<2.0dev,<3.0dev,==1.20.1,>=1.25.0,>=1.4.1 (from -r requirements.in (line 6))
Tried: 0.0.1, [...], 1.20.1, [...], 2.6.0
Skipped pre-versions: 2.0.0.dev0, 2.0.0.dev0, 2.0.0b1, 2.0.0b1
There are incompatible versions in the resolved dependencies:
  google-auth==1.20.1 (from -r requirements.in (line 6))
  google-auth<2.0dev,>=1.25.0 (from google-api-core[grpc]==1.31.5->firebase-admin==4.5.0->-r requirements.in (line 4))
  google-auth<3.0dev,>=1.25.0 (from google-cloud-storage==2.1.0->firebase-admin==4.5.0->-r requirements.in (line 4))
  google-auth (from google-auth-oauthlib==0.4.2->-r requirements.in (line 2))
  google-auth>=1.4.1 (from google-api-python-client==1.7.11->-r requirements.in (line 1))
  google-auth (from google-auth-httplib2==0.0.3->-r requirements.in (line 3))
  google-auth (from aiogoogle==1.1.13->-r requirements.in (line 5))

The new resolver from #1539 still barfs

$ pipx run --spec git+https://github.com/atugushev/pip-tools.git@new-resolver pip-compile --resolver=backtracking requirements.in
CRITICAL:pip._internal.resolution.resolvelib.factory:Cannot install google-auth==1.20.1 because these package versions have conflicting dependencies.
Traceback (most recent call last):
  File "/home/slate/.local/pipx/shared/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/slate/.local/pipx/shared/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: SpecifierRequirement('google-auth==1.20.1')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/slate/.local/pipx/shared/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
    result = self._result = resolver.resolve(
  File "/home/slate/.local/pipx/shared/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/slate/.local/pipx/shared/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 350, in resolve
    raise ResolutionImpossible(e.criterion.information)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('google-auth==1.20.1'), parent=None)]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/piptools/scripts/compile.py", line 481, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/home/slate/.local/pipx/.cache/d636a60d983d97c/lib/python3.9/site-packages/piptools/resolver.py", line 585, in resolve
    resolver.resolve(
  File "/home/slate/.local/pipx/shared/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 103, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Both modern and legacy pip -- 22.0.3 and 20.3.4 -- made funny noises and downloaded many redundant versions when running pip install -r requirements.in, but both eventually installed versions with no conflicts:

$ pip check
No broken requirements found.
$ pip freeze | grep -E 'google-api-python-client|google-auth-oauthlib|google-auth-httplib2|firebase-admin|aiogoogle|google-auth'
aiogoogle==1.1.13
firebase-admin==4.5.0
google-api-python-client==1.7.11
google-auth==1.20.1
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.4.2
$ pip freeze | grep -E 'google-api-core|google-cloud-storage'  # The 2nd-order dependencies
google-api-core==1.22.1
google-cloud-storage==1.40.0

(longer regex generated with: for p in $(cut -d= -f1 requirements.in); do echo -n "$p|"; done)

[ethanluoyc]

I have a similar issue but with a different set of requirements. Here's my requirements/base.in

absl-py
numpy
-f https://storage.googleapis.com/jax-releases/jax_releases.html
# Pinning the three versions to allow pip-compile-multi to work
jax[cuda]
tensorflow
tensorflow_probability
dm-acme[jax] @ git+https://github.com/deepmind/acme.git
# DeepMind JAX eco-system
chex
optax
rlax
dm-haiku
dm-reverb
dm-launchpad
ott-jax
dm-tree
# Environments
dm_control
wandb
ml_collections

dm-acme uses pinned dependencies for jax and tensorflow. The way I understand this is that if I do not pin or constrain the dependencies, then the resolver should figure out to use the pinned versions. However, I cannot get pip-compile to work in this case, giving the following error.

Could not find a version that matches jaxlib==0.3.10+cuda11.cudnn82,==0.3.5,>=0.1.37 (from dm-acme[jax]@ git+https://github.com/deepmind/acme.git->-r requirements/base.in (line 8))
Tried: 0.1.32, 0.1.40, 0.1.41, 0.1.42, 0.1.43, 0.1.44, 0.1.46, 0.1.50, 0.1.51, 0.1.52, 0.1.52+cuda100, 0.1.52+cuda101, 0.1.52+cuda102, 0.1.52+cuda110, 0.1.55, 0.1.55+cuda100, 0.1.55+cuda101, 0.1.55+cuda102, 0.1.55+cuda110, 0.1.56, 0.1.56+cuda100, 0.1.56+cuda101, 0.1.56+cuda102, 0.1.56+cuda110, 0.1.56+cuda111, 0.1.57, 0.1.57+cuda101, 0.1.57+cuda102, 0.1.57+cuda110, 0.1.57+cuda111, 0.1.58, 0.1.58+cuda101, 0.1.58+cuda102, 0.1.58+cuda110, 0.1.58+cuda111, 0.1.59, 0.1.59+cuda101, 0.1.59+cuda102, 0.1.59+cuda110, 0.1.59+cuda111, 0.1.60, 0.1.60, 0.1.60+cuda101, 0.1.60+cuda102, 0.1.60+cuda110, 0.1.60+cuda111, 0.1.61, 0.1.61, 0.1.61+cuda101, 0.1.61+cuda102, 0.1.61+cuda110, 0.1.61+cuda111, 0.1.61+cuda112, 0.1.62, 0.1.62, 0.1.62+cuda101, 0.1.62+cuda102, 0.1.62+cuda110, 0.1.62+cuda111, 0.1.62+cuda112, 0.1.63, 0.1.63, 0.1.63+cuda101, 0.1.63+cuda102, 0.1.63+cuda110, 0.1.63+cuda111, 0.1.63+cuda112, 0.1.64, 0.1.64, 0.1.64+cuda101, 0.1.64+cuda102, 0.1.64+cuda110, 0.1.64+cuda111, 0.1.64+cuda112, 0.1.65, 0.1.65, 0.1.65+cuda101, 0.1.65+cuda102, 0.1.65+cuda110, 0.1.65+cuda111, 0.1.65+cuda112, 0.1.66, 0.1.66, 0.1.66+cuda101, 0.1.66+cuda102, 0.1.66+cuda110, 0.1.66+cuda111, 0.1.67, 0.1.67, 0.1.67+cuda101, 0.1.67+cuda102, 0.1.67+cuda110, 0.1.67+cuda111, 0.1.68, 0.1.68, 0.1.68+cuda101, 0.1.68+cuda102, 0.1.68+cuda110, 0.1.68+cuda111, 0.1.69, 0.1.69, 0.1.69+cuda101, 0.1.69+cuda102, 0.1.69+cuda110, 0.1.69+cuda111, 0.1.70, 0.1.70, 0.1.70+cuda101, 0.1.70+cuda102, 0.1.70+cuda110, 0.1.70+cuda111, 0.1.71, 0.1.71, 0.1.71+cuda102, 0.1.71+cuda110, 0.1.71+cuda111, 0.1.72, 0.1.72, 0.1.72+cuda111, 0.1.73, 0.1.73, 0.1.73+cuda11.cudnn805, 0.1.73+cuda11.cudnn82, 0.1.74, 0.1.74, 0.1.74+cuda11.cudnn805, 0.1.74+cuda11.cudnn82, 0.1.75, 0.1.75, 0.1.75+cuda11.cudnn805, 0.1.75+cuda11.cudnn82, 0.1.76, 0.1.76, 0.1.76+cuda11.cudnn805, 0.1.76+cuda11.cudnn82, 0.3.0, 0.3.0, 0.3.0+cuda11.cudnn805, 0.3.0+cuda11.cudnn82, 0.3.2, 0.3.2, 0.3.2+cuda11.cudnn805, 0.3.2+cuda11.cudnn82, 0.3.5, 0.3.5, 0.3.5+cuda11.cudnn805, 0.3.5+cuda11.cudnn82, 0.3.7, 0.3.7, 0.3.7+cuda11.cudnn805, 0.3.7+cuda11.cudnn82, 0.3.8, 0.3.8+cuda11.cudnn805, 0.3.8+cuda11.cudnn82, 0.3.10, 0.3.10, 0.3.10+cuda11.cudnn805, 0.3.10+cuda11.cudnn82
There are incompatible versions in the resolved dependencies:
  jaxlib>=0.1.37 (from chex==0.1.3->-r requirements/base.in (line 10))
  jaxlib==0.3.10+cuda11.cudnn82 (from jax[cuda]==0.3.13->-r requirements/base.in (line 5))
  jaxlib>=0.1.37 (from optax==0.1.2->-r requirements/base.in (line 11))
  jaxlib==0.3.5 (from dm-acme[jax]@ git+https://github.com/deepmind/acme.git->-r requirements/base.in (line 8))
  jaxlib>=0.1.37 (from rlax==0.1.2->-r requirements/base.in (line 12))

[jondo]

I have a similar issue with Python 3.9.2, pip 22.1.2 and pip-tools 6.6.2.
A requirement.in with

tensorflow==2.5.3
tensorflow_similarity==0.14.2

yields:

$ pip-compile requirements.in
Could not find a version that matches typing-extensions>=3.10.0,~=3.7.4 (from tensorflow==2.5.3->-r requirements.in (line 20))
Tried: 3.6.2, 3.6.2, 3.6.2.1, 3.6.2.1, 3.6.5, 3.6.5, 3.6.6, 3.6.6, 3.7.2, 3.7.2, 3.7.4, 3.7.4, 3.7.4.1, 3.7.4.1, 3.7.4.2, 3.7.4.2, 3.7.4.3, 3.7.4.3, 3.10.0.0, 3.10.0.0, 3.10.0.1, 3.10.0.1, 3.10.0.2, 3.10.0.2, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0
There are incompatible versions in the resolved dependencies:
  typing-extensions>=3.10.0 (from bokeh==2.4.3->tensorflow_similarity==0.14.2->-r requirements.in (line 21))
  typing-extensions~=3.7.4 (from tensorflow==2.5.3->-r requirements.in (line 20))

For some reason pip-compile does not detect that typing-extensions 3.10.0 would fulfill the two restrictions.
I guess this will be solved by #1539? Is there any workaround (without falling back to pip install/pip freeze) that can be used while waiting for that?

[AndydeCleyre]

Is there any workaround (without falling back to pip install/pip freeze) that can be used while waiting for that?

Yes, you can add typing-extensions==3.10.0 to your requirements.in to give the resolver a hint.

[jondo]

Thank you, but this did not help, as a requirement.in with

typing-extensions==3.10.0
tensorflow==2.5.3

already yields

$ pip-compile -U requirements.in
Could not find a version that matches typing-extensions==3.10.0,~=3.7.4 (from -r requirements.in (line 20))
Tried: 3.6.2, 3.6.2, 3.6.2.1, 3.6.2.1, 3.6.5, 3.6.5, 3.6.6, 3.6.6, 3.7.2, 3.7.2, 3.7.4, 3.7.4, 3.7.4.1, 3.7.4.1, 3.7.4.2, 3.7.4.2, 3.7.4.3, 3.7.4.3, 3.10.0.0, 3.10.0.0, 3.10.0.1, 3.10.0.1, 3.10.0.2, 3.10.0.2, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0
There are incompatible versions in the resolved dependencies:
  typing-extensions==3.10.0 (from -r requirements.in (line 20))
  typing-extensions~=3.7.4 (from tensorflow==2.5.3->-r requirements.in (line 21))

(This time tested in a virtualenv with Python 3.8.10, pip 22.1.2 and pip-tools 6.6.2.)

Why does pip-compile not see 3.10.0~=3.7.4 as true?

[shaib]

Why does pip-compile not see 3.10.0~=3.7.4 as true?

Because it isn't. See https://peps.python.org/pep-0440/#compatible-release
~=3.7.4 is equivalent to >=3.7.4, ==3.7.*

[jondo]

Ah, sorry, my bad! I read the ~= for 'unequal', which is !=. So pip-compile was right in both of my examples.

[jondo]

I found a different workaround now! I noticed that tensorflow_similarity 0.14.2 does not specify any bokeh version, so I found a suitable one. My working requirement.in is now

tensorflow==2.5.3
tensorflow_similarity==0.14.2
bokeh==2.3.3

This will probably also be found by the new backtracking resolver #1539?

[atugushev]

This has been fixed in #1539 with the backtracking resolver, try pip-compile --resolver backtracking. The resolver is released as part of pip-tools v6.8.0. Please let us know if it doesn't resolve your issue. Thanks!