Cannot install python-language-server[all]

[renatocan]

I'm not sure this is the right place to report this. Anyway, I'm using pip-tools to manage my environments and I had a problem when I tried to upgrade to Python 3.10. After some analysis, I found out that the problem was related to the line python-language-server[all] in my requirements.in file. It works with Python 3.9.7 but it presents an error with Python 3.10.

Environment Versions

1. OS Type: Arch Linux 5.15.2-arch1-1
2. Python version: 3.10.0
3. pip version: pip 21.2.3
4. pip-tools version: pip-compile, version 6.4.0

Steps to replicate

1. Create a clean Python 3.10.0 virtualenv
2. Activate the virtualenv and install pip-tools
3. Create a requirements.in file with the line python-language-server[all]
4. Run pip-compile requirements.in

Expected result

Solve dependencies and create a requirements.txt file.

Actual result

(py310) ➜  a pip-compile requirements_py3.in           
Could not find a version that matches pycodestyle<2.7.0,<2.9.0,>=2.6.0,>=2.8.0 (from python-language-server[all]==0.36.2->-r requirements_py3.in (line 1))
Tried: 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.2.0, 2.2.0, 2.3.0, 2.3.0, 2.3.1, 2.3.1, 2.4.0, 2.4.0, 2.5.0, 2.5.0, 2.6.0, 2.6.0, 2.7.0, 2.7.0, 2.8.0, 2.8.0
Skipped pre-versions: 1.8.0.dev0, 1.8.0.dev0, 2.0.0a1, 2.0.0a1, 2.6.0a1, 2.6.0a1
There are incompatible versions in the resolved dependencies:
  pycodestyle<2.7.0,>=2.6.0 (from python-language-server[all]==0.36.2->-r requirements_py3.in (line 1))
  pycodestyle>=2.8.0 (from autopep8==1.6.0->python-language-server[all]==0.36.2->-r requirements_py3.in (line 1))
  pycodestyle<2.9.0,>=2.8.0 (from flake8==4.0.1->python-language-server[all]==0.36.2->-r requirements_py3.in (line 1))
...

Thanks!

[heavelock]

I think I stumbled on the same issue but in my case with Dask. As far as I understand there is a problem with resolving of dependency version of the packages that are supposed to be install via [all] switch.

❯ pip-compile requirements.in
Could not find a version that matches dask[complete]==2021.11.2,~=2.28.0 (from -r requirements.in (line 22))
Tried: 0.7.4.linux-x86_64, 0.7.5.linux-x86_64, 0.7.6.linux-x86_64, 0.8.1.macosx-10.5-x86_64, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.3.0, 0.4.0, 0.5.0, 0.6.0,
0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.6, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.0, 0.10.0, 0.10.0, 0.10.1, 0.10.1, 0.10.2, 0.10.2, 0.11.0, 0.
11.0, 0.11.1, 0.11.1, 0.12.0, 0.12.0, 0.13.0, 0.13.0, 0.14.0, 0.14.0, 0.14.1, 0.14.1, 0.14.2, 0.14.2, 0.14.3, 0.14.3, 0.15.0, 0.15.0, 0.15.1, 0.15.1, 0.15.2, 0.15.2, 0.15
.3, 0.15.3, 0.15.4, 0.15.4, 0.16.0, 0.16.0, 0.16.1, 0.16.1, 0.17.0, 0.17.0, 0.17.1, 0.17.1, 0.17.2, 0.17.2, 0.17.3, 0.17.3, 0.17.4, 0.17.4, 0.17.5, 0.17.5, 0.18.0, 0.18.0
, 0.18.1, 0.18.1, 0.18.2, 0.18.2, 0.19.0, 0.19.0, 0.19.1, 0.19.1, 0.19.2, 0.19.2, 0.19.3, 0.19.3, 0.19.4, 0.19.4, 0.20.0, 0.20.0, 0.20.1, 0.20.1, 0.20.2, 0.20.2, 1.0.0, 1
.0.0, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 1.1.2, 1.1.2, 1.1.3, 1.1.3, 1.1.4, 1.1.4, 1.1.5, 1.1.5, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.2.0, 2.2
.0, 2.3.0, 2.3.0, 2.4.0, 2.4.0, 2.5.0, 2.5.0, 2.5.2, 2.5.2, 2.6.0, 2.6.0, 2.7.0, 2.7.0, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 2.10.0, 2.10
.0, 2.10.1, 2.10.1, 2.11.0, 2.11.0, 2.12.0, 2.12.0, 2.13.0, 2.13.0, 2.14.0, 2.14.0, 2.15.0, 2.15.0, 2.16.0, 2.16.0, 2.17.0, 2.17.0, 2.17.1, 2.17.1, 2.17.2, 2.17.2, 2.18.0
, 2.18.0, 2.18.1, 2.18.1, 2.19.0, 2.19.0, 2.20.0, 2.20.0, 2.21.0, 2.21.0, 2.22.0, 2.22.0, 2.23.0, 2.23.0, 2.24.0, 2.24.0, 2.25.0, 2.25.0, 2.26.0, 2.26.0, 2.27.0, 2.27.0,
2.28.0, 2.28.0, 2.29.0, 2.29.0, 2.30.0, 2.30.0, 2020.12.0, 2020.12.0, 2021.1.0, 2021.1.0, 2021.1.1, 2021.1.1, 2021.2.0, 2021.2.0, 2021.3.0, 2021.3.0, 2021.3.1, 2021.3.1,
2021.4.0, 2021.4.0, 2021.4.1, 2021.4.1, 2021.5.0, 2021.5.0, 2021.5.1, 2021.5.1, 2021.6.0, 2021.6.0, 2021.6.1, 2021.6.1, 2021.6.2, 2021.6.2, 2021.7.0, 2021.7.0, 2021.7.1,
2021.7.1, 2021.7.2, 2021.7.2, 2021.8.0, 2021.8.0, 2021.8.1, 2021.8.1, 2021.9.0, 2021.9.0, 2021.9.1, 2021.9.1, 2021.10.0, 2021.10.0, 2021.11.0, 2021.11.0, 2021.11.1, 2021.
11.1, 2021.11.2, 2021.11.2
Skipped pre-versions: 0.13.0rc1, 0.13.0rc1
There are incompatible versions in the resolved dependencies:
  dask[complete]~=2.28.0 (from -r requirements.in (line 22))
  dask==2021.11.2 (from distributed==2021.11.2->dask[complete]==2.28.0->-r requirements.in (line 22))

As you can see, I specify in my requirements dask[complete]==2.28.0 but since [complete] switch installs also distributed, pip-tools is trying to resolve appropriate version for both of them. Unfortunately, it does not try to get the compatible version of distributed but rather the newest one.

Pinning distributed==2.28.0 made it possible for pip-compile to resolve the conflict.

[atugushev]

#1539 should fix the issue. Could you try it?

[heavelock]

Unfortunately, it did not fix my issue.

❯ pipx run --spec  git+https://github.com/atugushev/pip-tools.git@new-resolver pip-compile requirements.in

Could not find a version that matches dask[complete]==2.28.0,==2021.11.2 (from -r requirements.in (line 22))
Tried: 0.7.4.linux-x86_64, 0.7.5.linux-x86_64, 0.7.6.linux-x86_64, 0.8.1.macosx-10.5-x86_64, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.6, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.0, 0.10.0, 0.10.0, 0.10.1, 0.10.1, 0.10.2, 0.10.2, 0.11.0, 0.11.0, 0.11.1, 0.11.1, 0.12.0, 0.12.0, 0.13.0, 0.13.0, 0.14.0, 0.14.0, 0.14.1, 0.14.1, 0.14.2, 0.14.2, 0.14.3, 0.14.3, 0.15.0, 0.15.0, 0.15.1, 0.15.1, 0.15.2, 0.15.2, 0.15.3, 0.15.3, 0.15.4, 0.15.4, 0.16.0, 0.16.0, 0.16.1, 0.16.1, 0.17.0, 0.17.0, 0.17.1, 0.17.1, 0.17.2, 0.17.2, 0.17.3, 0.17.3, 0.17.4, 0.17.4, 0.17.5, 0.17.5, 0.18.0, 0.18.0, 0.18.1, 0.18.1, 0.18.2, 0.18.2, 0.19.0, 0.19.0, 0.19.1, 0.19.1, 0.19.2, 0.19.2, 0.19.3, 0.19.3, 0.19.4, 0.19.4, 0.20.0, 0.20.0, 0.20.1, 0.20.1, 0.20.2, 0.20.2, 1.0.0, 1.0.0, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 1.1.2, 1.1.2, 1.1.3, 1.1.3, 1.1.4, 1.1.4, 1.1.5, 1.1.5, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.2.0, 2.2.0, 2.3.0, 2.3.0, 2.4.0, 2.4.0, 2.5.0, 2.5.0, 2.5.2, 2.5.2, 2.6.0, 2.6.0, 2.7.0, 2.7.0, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 2.10.0, 2.10.0, 2.10.1, 2.10.1, 2.11.0, 2.11.0, 2.12.0, 2.12.0, 2.13.0, 2.13.0, 2.14.0, 2.14.0, 2.15.0, 2.15.0, 2.16.0, 2.16.0, 2.17.0, 2.17.0, 2.17.1, 2.17.1, 2.17.2, 2.17.2, 2.18.0, 2.18.0, 2.18.1, 2.18.1, 2.19.0, 2.19.0, 2.20.0, 2.20.0, 2.21.0, 2.21.0, 2.22.0, 2.22.0, 2.23.0, 2.23.0, 2.24.0, 2.24.0, 2.25.0, 2.25.0, 2.26.0, 2.26.0, 2.27.0, 2.27.0, 2.28.0, 2.28.0, 2.29.0, 2.29.0, 2.30.0, 2.30.0, 2020.12.0, 2020.12.0, 2021.1.0, 2021.1.0, 2021.1.1, 2021.1.1, 2021.2.0, 2021.2.0, 2021.3.0, 2021.3.0, 2021.3.1, 2021.3.1, 2021.4.0, 2021.4.0, 2021.4.1, 2021.4.1, 2021.5.0, 2021.5.0, 2021.5.1, 2021.5.1, 2021.6.0, 2021.6.0, 2021.6.1, 2021.6.1, 2021.6.2, 2021.6.2, 2021.7.0, 2021.7.0, 2021.7.1, 2021.7.1, 2021.7.2, 2021.7.2, 2021.8.0, 2021.8.0, 2021.8.1, 2021.8.1, 2021.9.0, 2021.9.0, 2021.9.1, 2021.9.1, 2021.10.0, 2021.10.0, 2021.11.0, 2021.11.0, 2021.11.1, 2021.11.1, 2021.11.2, 2021.11.2
Skipped pre-versions: 0.13.0rc1, 0.13.0rc1
There are incompatible versions in the resolved dependencies:
  dask[complete]==2.28.0 (from -r requirements.in (line 22))
  dask==2021.11.2 (from distributed==2021.11.2->dask[complete]==2.28.0->-r requirements.in (line 22))

Edit:
My bad. I forgot to add a feature flag to a command. Running command pipx run --spec git+https://github.com/atugushev/pip-tools.git@new-resolver pip-compile --resolver=2020 requirements.in resolves those dependencies properly.

[atugushev]

@heavelock, great! Thanks for the feedback 👍🏻

[renatocan]

I also confirm that running

pipx run --spec git+https://github.com/atugushev/pip-tools.git@new-resolver pip-compile --resolver=2020 requirements.in

resolves the dependencies properly.

Thanks!

[jdimatteo]

Hi is there a plan to merge #1539? I was using this but this branch is missing fixes like #1567

[atugushev]

@renatocan FYI I've rebase #1539 onto the latest master. It's still in the review process though, but nearly finished.

[atugushev]

This has been fixed in #1539 with the backtracking resolver, try pip-compile --resolver backtracking. The resolver is released as part of pip-tools v6.8.0. Please let us know if it doesn't resolve your issue. Thanks!