Poetry unable to find installation candidates for Nvidia c11 dependencies

[Alexander-Mark]

• Poetry version: 1.2.0
• Python version: poetry defaults to 3.8.14 from 3.10.6
• OS version and name: macOS 13 Ventura
• pyproject.toml: see in the file below:

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Hi there,

Having trouble using poetry with certain NLP libraries that have Nvidia c11 related dependencies. In this case I'm using EasyNMT for machine text translation, which has a dependency on torch.

The torch related issue #4231 was recently closed, so I wanted to open a new separate issue here.

I'm unable to install the 4 Nvidia c11 dependencies:

• nvidia-cublas-cu11 (11.10.3.66)
• nvidia-cuda-nvrtc-cu11 (11.7.99)
• nvidia-cuda-runtime-cu11 (11.7.99)
• nvidia-cudnn-cu11 (8.5.0.96)

Would appreciate any advice on whether this issue is due to my setup, or an issue with poetry. Please let me know if you're able to reproduce this issue, so I can review my environment accordingly. I have been able to lock and install these dependencies with pipenv, and then run the relevant EasyNMT code.

Thanks for the help!
Alex

pyproject.toml

[tool.poetry]
name = "Translation Project"
version = "1.0"
description = "Translating text"
authors = ["me <me@email.com>"]

[tool.poetry.dependencies]
python = "==3.8.14"
easynmt = "==2.0.2"

Checking works correctly:

poetry check

All set!

Locking works correctly:

poetry lock

The currently activated Python version 3.10.6 is not supported by the project (==3.8.14).
Trying to find and use a compatible version. 
Using python3 (3.8.14)
Creating virtualenv translation_porject-pPGuLrxo-py3.8 in /Users/alex/Library/Caches/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/ec/87/f26695307c0aa00e6938f5de795fc7f2c718a448b48d29a4c8c8dbf829d3/sentencepiece-0.1.97.tResolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz   Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  1Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  2Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  3Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  5Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  6Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  8Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3Resolving dependencies... (84.8s)

Writing lock file

Install does not work correctly:

poetry install -vvv

The currently activated Python version 3.10.6 is not supported by the project (==3.8.14).
Trying to find and use a compatible version. 
Trying python3
Using python3 (3.8.14)
Virtualenv translation_porject-pPGuLrxo-py3.8 already exists.
Using virtualenv: /Users/alex/Library/Caches/pypoetry/virtualenvs/translation_porject-pPGuLrxo-py3.8
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 11 installs, 0 updates, 0 removals, 20 skipped

  • Installing nvidia-cublas-cu11 (11.10.3.66): Pending...
  • Installing nvidia-cuda-nvrtc-cu11 (11.7.99): Pending...
  • Installing nvidia-cuda-runtime-cu11 (11.7.99): Pending...
  • Installing nvidia-cudnn-cu11 (8.5.0.96): Pending...
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
Creating new session for pypi.org
Creating new session for pypi.org
Skipping wheel nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cublas_cu11-11.10.3.66-py3-none-win_amd64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-py3-none-win_amd64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
  • Installing nvidia-cublas-cu11 (11.10.3.66): Failed

  Stack trace:

  5  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:250 in _execute_operation
        248│ 
      249│             try:
    → 250│                 result = self._do_execute_operation(operation)
      251│             except EnvCommandError as e:
      252│                 if e.e.returncode == -2:

  4  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:323 in _do_execute_operation
      321│             return 0
      322│ 
    → 323│         result: int = getattr(self, f"_execute_{method}")(operation)
      324│ 
      325│         if result != 0:

  3  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:443 in _execute_install
      441│ 
      442│     def _execute_install(self, operation: Install | Update) -> int:
    → 443│         status_code = self._install(operation)
      444│ 
      445│         self._save_url_reference(operation)

  2  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:477 in _install
      475│             archive = self._download_link(operation, Link(package.source_url))
      476│         else:
    → 477│             archive = self._download(operation)
      478│ 
      479│         operation_message = self.get_operation_message(operation)

  1  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:622 in _download
      620│ 
      621│     def _download(self, operation: Install | Update) -> Path:
    → 622│         link = self._chooser.choose_for(operation.package)
      623│ 
      624│         if link.yanked:

  RuntimeError

  Unable to find installation candidates for nvidia-cublas-cu11 (11.10.3.66)

  at /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/chooser.py:103 in choose_for
       99│ 
      100│             links.append(link)
      101│ 
      102│         if not links:
    → 103│             raise RuntimeError(f"Unable to find installation candidates for {package}")
      104│ 
      105│         # Get the best link
      106│         chosen = max(links, key=lambda link: self._sort_key(package, link))
      107│ 

[dimbleby]

no distribution is available for your platform. There is nothing poetry can do about this. This is not a bug.

https://pypi.org/project/nvidia-cublas-cu11/11.10.3.66/#files

[Alexander-Mark]

no distribution is available for your platform. There is nothing poetry can do about this. This is not a bug.

https://pypi.org/project/nvidia-cublas-cu11/11.10.3.66/#files

Ok cool -- what's confusing is that pipenv can install the exact same library with these dependencies (EasyNMT) on my platform (macOS) without any problems, so I thought it might be an issue since we recently switched to poetry. If you don't think it's an issue you can close this thanks, and we will try find a workaround.

[clintonroy]

If you're changing tools, I wouldn't change until everything is known to work. Does pipenv install exactly the same version? Often poetry tries to install the latest version, not the older one that works.

[spott]

does Pipenv install the cuda dependencies? or does Pipenv just install torch (without the different cuda dependencies)?

pip installs torch without those dependencies (as it should), I'm not sure why poetry is trying to add them. They aren't needed for torch on a Mac.

Any ideas?

[neersighted]

It sounds like torch may have different metadata in individual wheels. This is not supported by Poetry and can lead to surprising behavior, even though it is (obviously) possible. Instead, environment markers should be used to control whether dependencies are required or not on a platform-specific basis.

I don't have time to investigate this myself right now, but if anyone could confirm in this issue whether or not metadata is consistent across wheel tags, it would be appreciated.

[spott]

Between these two packages:

torch-1.13.0-cp39-cp39-manylinux1_x86_64.whl
torch-1.13.0-cp39-none-macosx_11_0_arm64.whl

The diff of the torch-1.13.0.dist-info/METADATA file is:

30a31,32
> License-File: LICENSE
> License-File: NOTICE
32,35d33
< Requires-Dist: nvidia-cuda-runtime-cu11 (==11.7.99)
< Requires-Dist: nvidia-cudnn-cu11 (==8.5.0.96)
< Requires-Dist: nvidia-cublas-cu11 (==11.10.3.66)
< Requires-Dist: nvidia-cuda-nvrtc-cu11 (==11.7.99)

So it looks like the metadata does change between versions...

[spott]

Ah!

From https://pypi.org/pypi/torch/json:

...
    "requires_dist": [
      "typing-extensions",
      "nvidia-cuda-runtime-cu11 (==11.7.99)",
      "nvidia-cudnn-cu11 (==8.5.0.96)",
      "nvidia-cublas-cu11 (==11.10.3.66)",
      "nvidia-cuda-nvrtc-cu11 (==11.7.99)",
      "opt-einsum (>=3.3) ; extra == 'opt-einsum'"
    ],
...

This is... unfortunate.

Looks like this is a new problem with 1.13.0 (1.12.1 installs without the Nvidia dependencies).

[neersighted]

Looks like a torch bug then -- this is the use case markers are intended for. Poetry does not support packages that rely on different metadata per wheel instead of using markers.

[jspaezp]

Seems to have been reported in pytorch:
pytorch/pytorch#88049

Edit [Nov 18 2022]:
For future reference pytorch/pytorch#88826
The fix seems to be merged with torch. I will update again when the wheels with the fix land in pypi

Edit [Dec 17 2022]:
1.31.1 has landed in pypi, and it works! setting pytorch requirement to "^1.31.1"

[eric79]

I believe the torch 1.13.1 version with the fixes in it has indeed landed in PyPI: https://pypi.org/project/torch/

[ccharp]

Is no one else still having problems with this? I get same error with 1.13.1.

[jangia]

Is no one else still having problems with this? I get same error with 1.13.1.

Same here

[mdagost]

Same.

[remy-radix]

For the new influx of people with the Mac M1 chip bumping into this thread, the current workaround is adding the correct wheel directly to your pyproject.toml:

[tool.poetry.dependencies]
python = "3.10"
torch = {url = "https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp310-cp310-linux_x86_64.whl"}

If you're developing locally but using docker for publishing, you can add markers as seen here.

[mdagost]

@remy-radix Thanks. The solution with the markers is nice, but I'm running into some odd issue with it where I have to do poetry install twice on the mac. Here is a simple pyproject.toml with just two packages--torch, and sentence-transformers, which depends on torch--to reproduce it with poetry 1.4.0:

[tool.poetry]
name = "torch-test"
version = "0.1.0"
description = ""
readme = "README.md"
packages = [{include = "torch_test"}]

[tool.poetry.dependencies]
python = "^3.10"
torch = [{markers = "sys_platform == 'macos'", url = "https://download.pytorch.org/whl/cpu/torch-1.13.1-cp310-none-macosx_11_0_arm64.whl"},
         {markers = "sys_platform == 'linux'", url = "https://download.pytorch.org/whl/torch-1.13.1-cp310-cp310-manylinux2014_aarch64.whl"}]
sentence-transformers = "^2.2.2"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

When I run poetry install --no-root --with lint in a fresh virtualenv on a mac, I see that it downloads the torch wheel but then gives this error:

Because no versions of sentence-transformers match >2.2.2,<3.0.0
 and sentence-transformers (2.2.2) depends on torch (>=1.6.0), sentence-transformers (>=2.2.2,<3.0.0) requires torch (>=1.6.0).
So, because no versions of torch match >=1.6.0
 and torch-test depends on sentence-transformers (^2.2.2), version solving failed.

It's like it doesn't recognize the torch version. If I then subsequently run poetry install --no-root --with lint a second time, everything works just fine, presumably because it's using the lock file.

However, if I do the same thing inside a docker container (without the lock file), it works fine the first time. And (thankfully) a poetry install inside the docker container using a lock file from the mac does work.

[heyajulia]

Make sure you’re not using Python 3.11 (or 3.12, for that matter). I just downgraded to 3.10, and I was able to successfully install PyTorch!

[kyaryunha]

torch 2.0.1 also shows the same problem :(
I used python 3.10.

[asmaier]

For python 3.9 I had to add

torch = "<1.13"

to my pyproject.toml file to avoid the issue.

[kyaryunha]

I solved it by using 2.0.0
If you want to use version 2, use 2.0.0.
I don't know if it was fixed later.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.