Instructions for installing PyTorch

[davidgilbertson]

• I have searched the issues of this repo and believe that this is not a duplicate.

Issue

As mentioned in issue #4231 there is some confusion around installing PyTorch with CUDA but it is now somewhat resolved. It still requires a few steps, and all options have pretty serious flaws. Below are two options that 'worked' for me, on Poetry version 1.2.0.

Option 1 - wheel URLs for a specific platform

• You will need to pick the specific wheels you want. These are listed here: https://download.pytorch.org/whl/torch_stable.html. E.g. if you want CUDA 11.6, Python 3.10 and Windows, search that page for cu116-cp310-cp310-win_amd64.whl to see the matches for torch, torchaudio and torchvision
• In your pyproject.toml file add the URLs like:

[tool.poetry.dependencies]
python = "^3.10"
numpy = "^1.23.2"
torch = { url = "https://download.pytorch.org/whl/cu116/torch-1.12.1%2Bcu116-cp310-cp310-win_amd64.whl"}
torchaudio = { url = "https://download.pytorch.org/whl/cu116/torchaudio-0.12.1%2Bcu116-cp310-cp310-win_amd64.whl"}
torchvision = { url = "https://download.pytorch.org/whl/cu116/torchvision-0.13.1%2Bcu116-cp310-cp310-win_amd64.whl"}

• Run poetry update. It will download a lot of data (many GB) and take quite some time. And this doesn't seem to cache reliably (at least, I've waited 30 minutes+ at 56 Mbps three separate times while troubleshooting this, for the exact same wheels)

Note that each subsequent poetry update will do another huge download and you'll see this message:

  • Updating torch (1.12.1+cu116 -> 1.12.1+cu116 https://download.pytorch.org/whl/cu116/torch-1.12.1%2Bcu116-cp310-cp310-win_amd64.whl)
  • Updating torchaudio (0.12.1+cu116 -> 0.12.1+cu116 https://download.pytorch.org/whl/cu116/torchaudio-0.12.1%2Bcu116-cp310-cp310-win_amd64.whl)
  • Updating torchvision (0.13.1+cu116 -> 0.13.1+cu116 https://download.pytorch.org/whl/cu116/torchvision-0.13.1%2Bcu116-cp310-cp310-win_amd64.whl)

Option 2 - alternate source

[tool.poetry.dependencies]
python = "^3.10"
numpy = "^1.23.2"
torch = { version = "1.12.1", source="torch"}
torchaudio = { version = "0.12.1", source="torch"}
torchvision = { version = "0.13.1", source="torch"}

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu116"
secondary = true

This seems to have worked (although I already had the packages installed) but it reports errors like Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pillow/, but I think they get installed anyway (maybe a better message would be "Can't access pillow at 'https://download.pytorch.org/whl/cu116', falling back to pypi")

Also, if you later go on to do, say poetry add pandas (a completely unrelated library) you'll get a wall of messages like:

Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pandas/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pandas/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pytz/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/python-dateutil/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/numpy/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pillow/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/requests/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/typing-extensions/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/certifi/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/urllib3/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/idna/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/charset-normalizer/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/python-dateutil/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/six/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/pytz/
Source (torch): Authorization error accessing https://download.pytorch.org/whl/cu116/six/

This happens with or without secondary = true in the source config.

Maintainers: please feel free to edit the text of this if I've got something wrong.

[dimbleby]

Failure to cache during resolution is covered by #2415, poetry's insistence on checking all sources for all packages is discussed at #5984

[variolam]

Solution 1 seems infeasible when working in a team with machines on different operating systems, due to the need of providing the complete URL of the wheel including operating system and exact version number.

Solution 2 seems to work, but it results in downloading every single PyTorch version that can found, independent on the operating system. I'm running Windows and the download looks like this:

A single installation takes around 15-20 minutes at ~250 Mbps.

[neersighted]

Poetry will always download wheels for every platform when you install -- this is because there is no other way to get package metadata from a repository using PEP 503's API.

[Queuecumber]

Can you elaborate a little on what metadata is needed and why downloading every conceivable version of a package yields that metadata? As mentioned this leads to a ~20min install for one package

[radoering]

Solution 1 seems infeasible when working in a team with machines on different operating systems, due to the need of providing the complete URL of the wheel including operating system and exact version number.

It's not convenient, but it should be feasible with multiple constraints dependencies.

[neersighted]

Poetry requires the package's core metadata aka the METADATA file (most critically this includes dependencies), as well as the bdist/sdist itself for hashing purposes. Note that PEP 658 is a standard for serving the METADATA file that is implementable by third-party repositories, and PEP 691 specifies a (potentially) richer JSON API (including hashes) that third-party repositories could likely implement.

However, Poetry is unlikely to grow support for these new APIs until PyPI does, and I think third party repos are unlikely to implement it before PyPI. Eventually support for these APIs will allow for feature and performance parity in Poetry between PyPI and third-party repositories. Until then, we are stuck with the legacy HTML API, which requires us to download every package when generating a lock file for the first time.

After your cache is warm you will not need to download again, and on other platforms you will only download the necessary files as the metadata is captured in the lock file.

[Queuecumber]

What I'm not understanding is that poetry knows I'm on Linux with python 3.8 but it still downloads

https://download.pytorch.org/whl/cu116/torch-1.12.1%2Bcu116-cp37-cp37-win_amd64.whl

Or does that wheel not contain the core metadata that is needed?

[Queuecumber]

Also there seems to be a second problem going on here unless I've misunderstood the documentation

I have this in my pyproject.toml

[[tool.poetry.source]]
name = "torchcu116"
url = "https://download.pytorch.org/whl/cu116"
default = false
secondary = true

i.e. secondary = true

Yet poetry is asking that repository for every package I try to install. I thought from the documentation that secondary meant it would go to pypi for any package unless specifically asked to go to that custom repository.

[neersighted]

What I'm not understanding is that poetry knows I'm on Linux with python 3.8 but it still downloads

https://download.pytorch.org/whl/cu116/torch-1.12.1%2Bcu116-cp37-cp37-win_amd64.whl

Or does that wheel not contain the core metadata that is needed?

Poetry constructs a universal lock file -- we write hashes to the lock file for all supported platforms. Thus on the first machine you generate a lock file, you will download a wheel for every supported platform. There is no way to write hashes to the lock file for those foreign/other platform versions without downloading them first.

If you want to reduce the scope of this a bit, you can tighten your Python constraint. There is a prototype of a new feature at #4956 (though it needs resurrection, design, and testing work) to add arbitrary markers to let a project reduce its supported platforms as an opt-in.

Also there seems to be a second problem going on here unless I've misunderstood the documentation

I have this in my pyproject.toml

[[tool.poetry.source]]
name = "torchcu116"
url = "https://download.pytorch.org/whl/cu116"
default = false
secondary = true

i.e. secondary = true

Yet poetry is asking that repository for every package I try to install. I thought from the documentation that secondary meant it would go to pypi for any package unless specifically asked to go to that custom repository.

I think it might be you misreading -- that is the intended and documented behavior. There is a proposal to introduce new repository types at #5984 (comment) as the current secondary behavior covers multiple use cases poorly, while being ideal for none of them.

[Queuecumber]

Poetry constructs a universal lock file

OK this makes sense now, thanks for the explanation, looking forward to that PR hopefully being merged eventually

I think it might be you misreading

I was misreading I see that this is intended behavior

the current secondary behavior covers multiple use cases poorly

I agree with that and I hope that these new repository types can be implemented

[ZetiMente]

Can't wait for option 2 to have good performance !

[neersighted]

Please 👍 on issues instead of commenting me too -- it keeps the notifications down and still shows interest. Thanks!

[voegtlel]

Probably a follow-up issue on the second option:
If I do poetry install on that, I get

Installing dependencies from lock file

Package operations: 50 installs, 0 updates, 0 removals

  • Installing wrapt (1.12.1): Failed

  KeyringLocked

  Failed to unlock the collection!

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/keyring/backends/SecretService.py:67 in get_preferred_collection
       63│             raise InitError("Failed to create the collection: %s." % e)
       64│         if collection.is_locked():
       65│             collection.unlock()
       66│             if collection.is_locked():  # User dismissed the prompt
    →  67│                 raise KeyringLocked("Failed to unlock the collection!")
       68│         return collection
       69│ 
       70│     def unlock(self, item):
       71│         if hasattr(item, 'unlock'):

I guess it tries to load that from the secondary repo as well, and expects to use the keyring due to the unauthorized thing?

[neersighted]

That's #1917 -- our use of keyring hits surprisingly many system configurations in which hard errors occur, and it needs some work.

[felix-ht]

@neersighted at least PyPI seems have started work on the new json APi

[neersighted]

Indeed, Poetry 1.2.2 relies on the new PEP 691 support. However, PEP 658 is the real blocker for better performance in third-party repos -- there is a long-running PR blocked on review and a rather combative contributor, but otherwise no major progress on that front.

[MaKaNu]

could you add a link to the blocked PR? I switched today to method 1 because method 2 took ages. Seems the meta servers are slowed down today... Dependency resolving which takes up to 4000 seconds for method 1 is also insane. And then it failed because I accidental copied 1.12.0 instead of 1.12.1 for the windows release. I really like the idea of poetry but this needs huge improvement.

[felix-ht]

We use neither these approaches.
My hacky solution to just install the base version of pytorch say 1.12.1 with poetry and then install the specific versions needed for the machines with pip in a makefile command.

As the gpu versions have the same dependencies as the base version this should be OK.

The big downsides are

• download both torch versions
• lockfile does not mach installed version (of pytorch)
• a change in the pytorch version requires and update to the makefile as well as the pyproject.toml
• any poetry action like poetry add will reinstall the base version
• breaks Packaging - usually not a concern for ml projects

The big upside is that it is very easy to create make scripts for different machines - and that its pretty fast (very important for ci/cd)
Example:

[tool.poetry.dependencies]
python = "^3.10"
numpy = "^1.23.2"
torch = "1.12.1"
torchvision = "0.13.1"

install_cu116:
	poetry install
	poetry run pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 -f https://download.pytorch.org/whl/torch_stable.html

[timothyjlaurent]

install_cu116:
	poetry install
	poetry run pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 -f https://download.pytorch.org/whl/torch_stable.html

this has essentially been our approach, but in Dockerfiles, gated by a build arg

ARG TORCH_ARCH="cpu"
#ARG TORCH_ARCH="cu113"

RUN  poetry install -vvv --no-root --no-dev \
    && pip install -U wheel torch==1.12.1+${TORCH_ARCH} torchvision==0.13.1+${TORCH_ARCH} -f https://download.pytorch.org/whl/torch_stable.html \
    && pip uninstall poetry -y \
    && rm -rf ~/.config/pypoetry \
    && rm -rf /root/.cache/pip

This also allows installing the CPU version which is smaller a smaller package that lacks the CUDA drivers that come in the normal pytorch package from pypi. They help to slim the images down.

[brochier]

Hi ! Do you know if it's possible to specify two different optional versions of torch in the pyproject.toml ? I would like to use a cpu version locally and a gpu version on a distant server. You can have a look at an example in this stack overflow post.

[neersighted]

That is something not unlike #5222; the consensus has been that as Poetry is an interoperable tool, no functionality will be added to the core project to support this until there is a standards-based method. A plugin can certainly support this with some creativity and would be the immediate "I want Poetry to support this" use case solution in my mind.

[robinbrochier]

#5222

Thanks for your answer, do you have any such plugin in mind ?

[neersighted]

I don't have any links at hand, but building on top of Light the Torch has been discussed. But if you mean if I know anyone is working on one, no, not that I am aware of.

[arthur-st]

For as long as pip install torch does Just Work ™️, having to put any effort at all into installing torch via poetry will be a vote against using it in commercial ML work that requires torch or similar hardware-bound components. Assuming that one is open to consider poetry, conda, and pipenv as functionally equivalent options, of course.

[neersighted]

I don't see that as a Poetry problem -- the issue is that there is an unmet packaging need, and no one from the ML world is working with the PyPA to define how to handle this robustly/no one from the PyPA has an interest in solving it.

Likewise, there is no interest in Poetry in supporting idiosyncratic workarounds for a non-standard and marginally compatible ecosystem; we'll be happy to implement whatever standards-based process evolves to handle these binary packages, but in the mean time any special-casing and package-specific functionality belong in a plugin and not Poetry itself.

[aa956]

Do I understand correctly there is no easy way to use poetry for projects using pytorch and expecting to get cross-platform GPU acceleration?

Started a new project, thought that something better than pip would be nice to use for once.

Run through poetry introduction and basic usage docs, installed poetry, created a new project, added click.

Very nice so far.

Then tried to add pytorch. Looked at the documentation here https://python-poetry.org/docs/repositories/

Got the following in pyproject.toml:

[tool.poetry.dependencies]
python = "^3.11"
click = ">=8.1.7"

[[tool.poetry.source]]
name = "torch-cu121"
url = "https://download.pytorch.org/whl/cu121"
priority = "supplemental"

[[tool.poetry.source]]
name = "torch-cu118"
url = "https://download.pytorch.org/whl/cu118"
priority = "supplemental"

[[tool.poetry.source]]
name = "torch-rocm56"
url = "https://download.pytorch.org/whl/rocm5.6"
priority = "supplemental"

[[tool.poetry.source]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "supplemental"

And got the following after poetry add --source torch-cu121 torch torchvision on windows:

> poetry add --source torch-cu121 torch torchvision
Using version ^2.1.2+cu121 for torch
Using version ^0.16.2+cu121 for torchvision

Updating dependencies
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-linux_x86_64.whl  11% (0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-linux_x86_64.whl  29% (0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-linux_x86_64.whl  47% (0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-linux_x86_64.whl  63% (0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-linux_x86_64.whl  80% (0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-linux_x86_64.whl  99% (0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-win_amd64.whl  14% (0.7s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-win_amd64.whl  37% (0.8s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-win_amd64.whl  59% (0.9s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-win_amd64.whl  75% (1.0s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp310-cp310-win_amd64.whl  99% (1.1s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-linux_x86_64.whl   9% (1
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-linux_x86_64.whl  27% (1
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-linux_x86_64.whl  45% (1
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-linux_x86_64.whl  60% (1
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-linux_x86_64.whl  77% (2
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-linux_x86_64.whl  94% (2
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-win_amd64.whl   2% (2.2s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-win_amd64.whl  25% (2.3s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-win_amd64.whl  48% (2.4s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-win_amd64.whl  65% (2.5s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-win_amd64.whl  82% (2.6s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp311-cp311-win_amd64.whl  91% (2.7s
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl   6% (2.8
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  12% (2.9
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  21% (3.0
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  33% (3.1
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  42% (3.2
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  50% (3.3
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  61% (3.4
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  71% (3.5
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  80% (3.6
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  89% (3.7
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp38-cp38-linux_x86_64.whl  99% (3.8
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp39-cp39-linux_x86_64.whl  19% (5.3
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp39-cp39-linux_x86_64.whl  37% (5.4
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp39-cp39-linux_x86_64.whl  52% (5.5
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp39-cp39-linux_x86_64.whl  70% (5.6
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torchvision-0.16.2%2Bcu121-cp39-cp39-linux_x86_64.whl  89% (5.8
Resolving dependencies... Downloading https://download.pytorch.org/whl/cu121/torch-2.1.2%2Bcu121-cp310-cp310-linux_x86_64.whl  29% (65.3s)
^C^C^C

Killed poetry as soon as seen what it tries to download so not sure if it would have tried to install all of the downloads too.

Expected something at least not a lot worse than plain pip - 40-50 seconds to install, nothing downloaded, not even the 2.5Gb wheels that were installed as everything is cached after previous installs of various projects that use torch for this OS/user.

[mathewcohle]

Using following workaround around the setup described in #6409 (comment):

[tool.poetry.dependencies]
torch = {version = "^2.1.2", source = "pytorch-cpu", markers = "extra!='cuda'" }

[tool.poetry.group.remote]
optional = true

[tool.poetry.group.remote.dependencies]
torch = {version = "^2.1.2", source = "pytorch-cu121", markers = "extra=='cuda'"}

[tool.poetry.extras]
cuda = []

[[tool.poetry.source]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[[tool.poetry.source]]
name = "pytorch-cu121"
url = "https://download.pytorch.org/whl/cu121"
priority = "explicit"

and then installing the dependencies:

poetry install # to get CPU version
poetry install -E cuda --with remote # to get GPU version

Seems like poetry.lock is properly generated and the setup is working.

Hope there will be better solution in the near future (:

[dimbleby]

a couple of relevant fixes recently

• torch had regressed their index such that it didnt provide hashes (since poetry needs the hash values for its lockfile, this is what causes poetry to download all those wheels during locking). They seem to have solved that again, hopefully it sticks this time

• feature: add new "lazy-wheel" config option #8815 allows poetry to get wheel metadata without downloading the whole wheel

the upshot is that the sequence given in #6409 (comment) when run with the latest master poetry results in locking succeeding in roughly 5 seconds (with downloads of roughly no gigabytes).

of course there still are a lot of wheels to download during installation, but there really is nothing much to be done about that.

[aa956]

a couple of relevant fixes recently

Thank you, install worked as expected after poetry update to master branch!

[slashtechno]

I tried the @mathewcohle's approach but I get the following with poetry install:
This is on Windows with Python 3.11.5

However, I get the following, and think it might be related.

  RuntimeError

  Unable to find installation candidates for nvidia-cudnn-cu11 (8.7.0.84)

  at ~\AppData\Roaming\pypoetry\venv\Lib\site-packages\poetry\installation\chooser.py:76 in choose_for
       72│
       73│             links.append(link)
       74│
       75│         if not links:
    →  76│             raise RuntimeError(f"Unable to find installation candidates for {package}")
       77│
       78│         # Get the best link
       79│         chosen = max(links, key=lambda link: self._sort_key(package, link))
       80│

What Python version are you using, and on what platform?

EDIT: Upon further testing, the error continues even when no operations are being taken with torch. It seems it was caused by TensorFlow, which was also a dependency. In addition to installing Torch with GPU capability, is it possible to install TensorFlow with CUDA (tensorflow = {version = "^2.14.0", extras = ["and-cuda"]}) when using -E cuda but otherwise, install it normally (tensorflow = {version = "^2.14.0"})?

[david-waterworth]

For me @mathewcohle's approach results in neither version being installed when I type poetry install -E cuda --with group - despite not being installed it tells me No dependencies to install or update

The only difference I see is I have two groups one which uses torch cpu and the other torch gpu. Also I have a private pypi repo as well (AWS Code Artifact) which contains torch, so I wonder if the lock file is being incorrectly generated because many packages I use have torch as a dependency. I try to explicitly specify torch as a dependency first (before say transformers) but perhaps this is failing.

I'm trying to create different environments during my build process. I have a docker file which I construct using one group with torch cpu, a local experiments group where I want torch on my workstation with cuda, and a build group which doesn't require either version (it just registers pipeline steps in sagemaker using the built docker image(s).

[MaKaNu]

I tried the @mathewcohle's approach but I get the following with poetry install: This is on Windows with Python 3.11.5

...

What Python version are you using, and on what platform?

EDIT: Upon further testing, the error continues even when no operations are being taken with torch. It seems it was caused by TensorFlow, which was also a dependency. In addition to installing Torch with GPU capability, is it possible to install TensorFlow with CUDA (tensorflow = {version = "^2.14.0", extras = ["and-cuda"]}) when using -E cuda but otherwise, install it normally (tensorflow = {version = "^2.14.0"})?

If not aware I want to mention that Tensorflow does not support native Windows anymore with cuda support.

[david-waterworth]

A follow up on my comment, the reason my attempt failed was I wanted to add torch to optional groups, and I wanted to be able to specify a different architecture per group (i.e. I have a notebooks group for local experiments, a docker group for building AWS pipeline steps and a build group which runs my build scripts, I don't want to install any version of torch on the build server, the docker container must use the specific version of cuda shipped in the AWS container, and for local experiments I want to select whatever is appropriate for my workstation.)

So far I've not been able to do this, the closest I've got is to install torch+cpu by default, or torch+cuXXX as an extra.

[[tool.poetry.source]]
name = "torch+cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[[tool.poetry.source]]
name = "torch+cu117"
url = "https://download.pytorch.org/whl/cu117"
priority = "explicit"

[tool.poetry.extras]
cuda = ["torch"]

[tool.poetry.dependencies]
python = "^3.10"
torch = [
    {version = "2.0.0", markers = "extra != 'cuda'", source = "torch+cpu"},
    {version = "2.0.0", markers = "extra == 'cuda'", source = "torch+cu117", optional = true}
    ]

Is there any way of marking both torch versions optional, and then using extra's / markers to install either one or the other or neither? The issue I see here is there are multiple other groups, some of which contain packages that rely on torch. So I can see where the complexity is. In my case my groups are mutually exclusive but there's no way of expressing that contraint in poetry.

[david-waterworth]

So the issue I have is each time I run the install command it swaps between cuda and cpu, i.e.

>poetry install -E cuda

Installing dependencies from lock file

Package operations: 0 installs, 1 update, 0 removals

  • Updating torch (2.0.0+cpu -> 2.0.0+cu117)

>poetry install -E cuda
Installing dependencies from lock file

Package operations: 0 installs, 1 update, 0 removals

  • Downgrading torch (2.0.0+cu117 -> 2.0.0+cpu)

>poetry install -E cuda

Installing dependencies from lock file

Package operations: 0 installs, 1 update, 0 removals

  • Updating torch (2.0.0+cpu -> 2.0.0+cu117)

etc

[QuentinSoubeyranAqemia]

@david-waterworth This has already been found out above in this comment. To be fair I'm not quite clear on why. I suspect extra != doesn't work as we'd like.

One thing I have yet to try is to have all torch version be marked as optional, and filtered on a specific extra (cpu, cuXX, etc...). Any dependency that then depends on torch would also need be optional. If necessary, they can also be filtered on the extra to sync cpu/gpu/etc... capabilities.
The cpu, cuXX etc... extra must not be empty, otherwise poetry seems to ignore it. In this setups it contains torch and all additional package that depends on torch, so that shouldn't be a problem.

If this doesn't work, then I don't understand what the markers = "extra == 'cuda'" does, and this is not documented in either the PEP nor poetry's own documentation to the best of my knowledge.

[DWarez]

@QuentinSoubeyranAqemia I also think that "extra != 'foo'" is not working as intended.

I am trying something like:

[tool.poetry.group.remote_cpu]
optional = true

[tool.poetry.group.remote_cuda]
optional = true

[tool.poetry.group.remote_mps]
optional = true

[tool.poetry.group.remote_cpu.dependencies]
torch = {version = "^2.2.0", source = "pytorch-cpu", markers = "extra=='cpu' and extra!='mps' and extra!='cuda'"}

[tool.poetry.group.remote_cuda.dependencies]
torch = {version = "^2.2.0", source = "pytorch-cu121", markers = "extra=='cuda' and extra!='mps' and extra!='cpu'"}

[tool.poetry.group.remote_mps.dependencies]
torch = {version = "^2.2.0", markers = "extra=='mps' and extra!='cuda' and extra!='cpu'"}

[tool.poetry.extras]
cpu = ["cpu"]
cuda = ["cuda"]
mps = ["mps"]

However, this seems not to work. It really looks like the extra!='foo' has no impact on the install, even if it is required to make the constraints work.

[creat89]

The issue with the marker extra is well known, see #7748

[david-waterworth]

@DWarez I think also the value extra's are supposed to be the package name as well aren't they - i.e.

[tool.poetry.extras]
cpu = ["torch"]
cuda = ["torch"]
mps = ["torch"]

I'm not totally sure how it's supposed to work, or if it's working as expected and we're abusing it. Also so far the only way I've got this close to working is to add torch as a main dependency. Adding it to multiple optional groups always seems to in poetry check reporting

Error: Cannot find dependency "torch" for extra "cpu" in main dependencies.
Error: Cannot find dependency "torch" for extra "cuda" in main dependencies.

I was thinking it would also be nice if upstream (torch) was refactored so there was a base package (i.e. torch) and optional packages for each accelerator (i.e. torch_cuda, torch_rocm etc.) so they could all be installed from the same repo (i.e. pip install torch or pip install torch[torch_cuda] etc. I'm not at all sure this would fix the issue of being able to instruct poetry to install torch for a specific accelerator, or not install it at all, depending on certain runtime parameters though.

[DWarez]

@david-waterworth I tried a lot of different tricks, also the one you just mentioned, but still I cannot make things work when trying configure for cpu, cuda and mps. The trick described in #6409 (comment) works, however it seems like that when defining multiple conditions in the markers, some of them (if not all) are ignored.

e.g.
markers = "extra!='cuda' and extra!='mps'" doesn't work at all.

[QuentinSoubeyranAqemia]

@DWarez Note that groups and extras are not the same thing, and you are mixing those up.

• extra(s) is a mecanism to opt-in to additional dependencies of a package, and are part of a distribution package's metadata. They are specified in the Python Packaging Authority metadata spec.
• groups are a poetry-specific way to have dependencies that only poetry can see, for the doc says:

Dependency groups, other than the implicit main group, must only contain dependencies you need in your development process. Installing them is only possible by using Poetry.

To declare a set of dependencies, which add additional functionality to the project during runtime, use extras instead. Extras can be installed by the end user using pip.

Markers are a PyPA specification and thus aren't related to the groups you define.

---

@creat89 Thank you for #7748, it escaped my radar and this seems to be the relevant piece.

[slashtechno]

Would this be the correct usage of extras and groups? Curious as I'm facing the same problem wherein poetry install -E cuda --with gpu switches between PyTorch versions and am unsure how to fix it.

[tool.poetry.dependencies]
torch = {version = "2.1.*", source = "pytorch-cpu", markers = "extra!='cuda'" }
tensorflow = {version = "^2.14.0", markers = "extra!='cuda'"}
...
[tool.poetry.group.gpu]
optional = true

[tool.poetry.group.gpu.dependencies]
torch = {version = "2.1.*", source = "pytorch-cu121", markers = "extra=='cuda'"}
tensorflow = {version = "^2.14.0", extras = ["and-cuda"], markers = "extra=='cuda'"}

[tool.poetry.extras]
# Might be better to rename this to nocpu since it's more accurate
cuda = []

[[tool.poetry.source]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[[tool.poetry.source]]
name = "pytorch-cu121"
url = "https://download.pytorch.org/whl/cu121"
priority = "explicit"

[peterdudfield]

adding in torch = {version = "2.1.*", source = "pytorch-cpu",} and \

[[tool.poetry.source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[[tool.poetry.source]]
name = "torch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "supplemental"

Solved my problem, of trying to just install cpu torch

[dimbleby]

pytorch say

PyTorch should work well with poetry in 2.1 and above. If this is not the case, please do not hesitate to file a separate issue

[alihaskar]

you can use this command and run pip from poetry:

poetry run pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

[RGX650]

My working pyproject if it can help anybody... (Poetry 1.8.2)
No circular dependencies.

I edited my pyproject.toml with a .ps1 script then:

poetry lock --no-update
poetry debug resolve --tree -vvvv
poetry lock --no-update
poetry install --no-root --only main
poetry show -vvvv
poetry export -f requirements.txt --output requirements.txt --without-hashes

i am using a local pypi server, but you can use the configurated source(s) regarding your needs:

[tool.poetry]
package-mode = false
name = "poetryghost"
version = "0.1.0"
description = "PoetryPackageManager"
authors = ["ZRGX650"]
license = "proprietary"
readme = "README.md"

[tool.poetry.dependencies]
python = "3.10.*"

Cython = { path = "C:/AI/POETRYGHOST/Cython-3.0.9-cp310-cp310-win_amd64.whl" }

triton = { path = "C:/AI/POETRYGHOST/triton-2.1.0-cp310-cp310-win_amd64.whl" }

nvidia_cudnn_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cudnn_cu12-8.9.7.29-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cublas_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cublas_cu12-12.1.3.1-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_cupti_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_cupti_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_runtime_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_runtime_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_nvrtc_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_nvcc_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_nvcc_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_sanitizer_api_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_sanitizer_api_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cufft_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cufft_cu12-11.0.2.54-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_curand_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_curand_cu12-10.3.2.106-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cusolver_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cusolver_cu12-11.4.5.107-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cusparse_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cusparse_cu12-12.1.0.106-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_npp_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_npp_cu12-12.1.1.14-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_nvjitlink_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_nvjitlink_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_nvjpeg_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_nvjpeg_cu12-12.1.1.14-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_nvtx_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_nvtx_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_opencl_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_opencl_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_cccl_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_cccl_cu12-12.1.109-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
nvidia_cuda_cuxxfilt_cu12 = { path = "C:/AI/POETRYGHOST/nvidia_cuda_cuxxfilt_cu12-12.1.105-py3-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}

cuda_python = { path = "C:/AI/POETRYGHOST/cuda_python-12.1.0-cp310-cp310-win_amd64.whl" }
pycuda = { path = "C:/AI/POETRYGHOST/pycuda-2024.1.tar.gz" }
cupy_cuda12x = { path = "C:/AI/POETRYGHOST/cupy_cuda12x-13.0.0-cp310-cp310-win_amd64.whl" }

torch = { path = "C:/AI/POETRYGHOST/torch-2.2.1+cu121-cp310-cp310-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
torchaudio = { path = "C:/AI/POETRYGHOST/torchaudio-2.2.1+cu121-cp310-cp310-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
torchvision = { path = "C:/AI/POETRYGHOST/torchvision-0.17.1+cu121-cp310-cp310-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}

torchdata = { path = "C:/AI/POETRYGHOST/torchdata-0.7.1-cp310-cp310-win_amd64.whl" }
torchtext = { path = "C:/AI/POETRYGHOST/torchtext-0.17.1+cpu-cp310-cp310-win_amd64.whl" }
torchmetrics = { path = "C:/AI/POETRYGHOST/torchmetrics-1.0.3-py3-none-any.whl" }
torchserve = { path = "C:/AI/POETRYGHOST/torchserve-0.10.0-py3-none-any.whl" }
lit = { path = "C:/AI/POETRYGHOST/lit-18.1.0rc4.tar.gz" }
depyf = { path = "C:/AI/POETRYGHOST/depyf-0.14.1-py3-none-any.whl" }
bitsandbytes = { path = "C:/AI/POETRYGHOST/bitsandbytes-0.43.0-py3-none-win_amd64.whl" }

xformers = { path = "C:/AI/POETRYGHOST/xformers-0.0.26.dev769-cp310-cp310-win_amd64.whl" }

flash_attn = { path = "C:/AI/POETRYGHOST/flash_attn-2.5.2+cu122torch2.2.0cxx11abiFALSE-cp310-cp310-win_amd64.whl" }

tensorboard = { path = "C:/AI/POETRYGHOST/tensorboard-2.16.2-py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
torch_tb_profiler = { path = "C:/AI/POETRYGHOST/torch_tb_profiler-0.4.3-py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}

onnx = { path = "C:/AI/POETRYGHOST/onnx-1.17.0-cp310-cp310-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
onnxruntime_gpu = { path = "C:/AI/POETRYGHOST/onnxruntime_gpu-1.17.1-cp310-cp310-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
tf2onnx = { path = "C:/AI/POETRYGHOST/tf2onnx-1.16.1-py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
onnx2tf = { path = "C:/AI/POETRYGHOST/onnx2tf-1.19.11-py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
onnx2onnx = { path = "C:/AI/POETRYGHOST/onnx2onnx-0.2.4-py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
onnx_extended = { path = "C:/AI/POETRYGHOST/onnx_extended-0.2.4-cp310-cp310-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}

tensorrt = { path = "C:/AI/POETRYGHOST/tensorrt-8.6.1-cp310-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
tensorrt_lean = { path = "C:/AI/POETRYGHOST/tensorrt_lean-8.6.1-cp310-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
tensorrt_dispatch = { path = "C:/AI/POETRYGHOST/tensorrt_dispatch-8.6.1-cp310-none-win_amd64.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
graphsurgeon = { path = "C:/AI/POETRYGHOST/graphsurgeon-0.4.6-py2.py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
onnx_graphsurgeon = { path = "C:/AI/POETRYGHOST/onnx_graphsurgeon-0.3.12-py2.py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}
uff = { path = "C:/AI/POETRYGHOST/uff-0.6.9-py2.py3-none-any.whl", markers = 'platform_system == "Windows" and sys_platform == "win32"'}

stable_fast = { path = "C:/AI/POETRYGHOST/stable_fast-1.0.4+torch221cu121-cp310-cp310-win_amd64.whl" }

[[tool.poetry.source]]
name = "PyPI"
priority = "primary"

[[tool.poetry.source]]
name = "GHOSTREPO"
url = "http://localhost:8080/simple"
priority = "primary"

[[tool.poetry.source]]
name = "GHOSTWHL"
url = "file:///C:/AI/POETRY/WHEELS"
priority = "explicit"

[[tool.poetry.source]]
name = "pytorch-cu121"
url = "https://download.pytorch.org/whl/cu121"
priority = "supplemental"

[[tool.poetry.source]]
name = "pytorch-nightly-cu121"
url = "https://download.pytorch.org/whl/nightly/cu121"
priority = "supplemental"

[[tool.poetry.source]]
name = "nvidia-json"
url = "https://developer.download.nvidia.com/compute/cuda/redist/redistrib_12.1.1.json"
priority = "supplemental"

[[tool.poetry.source]]
name = "nvidia-pypi"
url = "https://pypi.nvidia.com/simple"
priority = "supplemental"

[[tool.poetry.source]]
name = "nvidia-ngc"
url = "https://pypi.ngc.nvidia.com/simple"
priority = "supplemental"

[[tool.poetry.source]]
name = "xformers-cu121"
url = "https://download.pytorch.org/whl/cu121/xformers"
priority = "explicit"

[[tool.poetry.source]]
name = "onnxruntime-cuda-121"
url = "https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple"
priority = "explicit"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[PyroGenesis]

@alihaskar You could install any package using poetry run pip install but it wouldn't add it the dependency to your pyproject.toml file (which means cloning the repo would not give the same environment on running poetry install) and poetry will uninstall pytorch again when you run poetry install --sync

@RGX650 You are installing pytorch using a wheel already in your system, which is already mentioned in the issue OP and does not go through dependency resolution which is the issue many people have here.

By the way, this comment is still an issue for pytorch 2.2.2. Resolves properly for pytorch 2.2.1 though, so maybe the pytorch index has regressed again as @dimbleby mentioned here

EDIT: Confirmed there are no hashes for 2.2.2 by checking it personally. Has anyone tried using solver.lazy-wheel to mitigate this issue?

[dimbleby]

lazy wheel cannot help with missing hashes, missing hashes are missing anyway.

talking about this here does no good, you should report it to torch

[RGX650]

@PyroGenesis,
while using my local GHOSTREPO or the configurated source "pytorch-cu121"
it seems that is actually does install and resolve the necessary dependencies...
(maybe i am wrong, please tell me)

When removing the cuda libraries my pyproject.toml and using using source "pytorch-cu121",
and changing Torch, torchaudio and torchvision lines with structure like so:

PackageName = { version = "PackageVersion", source = 'pytorch-cu121', markers = "platform_system == 'Windows' and sys_platform == 'win32'" }

After poetry install:

(envsystem) C:\AI\POETRY\PROJECTS\poetryghost\envsystem\Scripts>poetry show

absl-py
appdirs
astor
bitsandbytes
certifi
charset-normalizer
colorama
coloredlogs
cuda-python
cupy-cuda12x
Cython
depyf
dill
einops
fastrlock
filelock
flash_attn
flatbuffers
fsspec
graphsurgeon
grpcio
humanfriendly
idna
jinja2
lightning-utilities
lit
mako
markdown
markupsafe
mpmath
networkx
ninja
numpy
nvidia-cublas-cu12
nvidia-cuda-cccl-cu12
nvidia-cuda-cupti-cu12
nvidia-cuda-cuxxfilt-cu12
nvidia-cuda-nvcc-cu12
nvidia-cuda-nvrtc-cu12
nvidia-cuda-opencl-cu12
nvidia-cuda-runtime-cu12
nvidia-cuda-sanitizer-api-cu12
nvidia-cudnn-cu12
nvidia-cufft-cu12
nvidia-curand-cu12
nvidia-cusolver-cu12
nvidia-cusparse-cu12
nvidia-npp-cu12
nvidia-nvjitlink-cu12
nvidia-nvjpeg-cu12
nvidia-nvtx-cu12
onnx
onnx-extended
onnx-graphsurgeon
onnx2onnx
onnx2tf
onnxruntime-gpu
packaging
pandas
pillow
platformdirs
protobuf
psutil
pycuda
pyreadline3
python-dateutil
pytools
pytz
requests
scipy
setuptools
six
stable-fast
sympy
tabulate
tensorboard
tensorboard-data-server
tensorrt
tensorrt-dispatch
tensorrt-lean
tf2onnx
torch
torch-tb-profiler
torchaudio
torchdata
torchmetrics
torchserve
torchtext
torchvision
tqdm
triton
typing-extensions
tzdata
uff
urllib3
werkzeug
wheel
xformers

(envsystem) C:\AI\POETRY\PROJECTS\poetryghost\envsystem\Scripts>pipdeptree

black==24.1a1
├── click [required: >=8.0.0, installed: 8.1.7]
│ └── colorama [required: Any, installed: 0.4.6]
├── mypy-extensions [required: >=0.4.3, installed: 1.0.0]
├── packaging [required: >=22.0, installed: 23.2]
├── pathspec [required: >=0.9.0, installed: 0.12.1]
├── platformdirs [required: >=2, installed: 4.1.0]
├── tomli [required: >=1.1.0, installed: 2.0.1]
└── typing-extensions [required: >=4.0.1, installed: 4.9.0]
cuda-python==12.3.0
Cython==3.0.9
graphsurgeon==0.4.6
ninja==1.11.1.1
onnx-graphsurgeon==0.3.12
├── numpy [required: Any, installed: 1.26.3]
└── onnx [required: Any, installed: 1.15.0]
├── numpy [required: Any, installed: 1.26.3]
└── protobuf [required: >=3.20.2, installed: 3.20.3]
pip-system-certs==4.0
└── wrapt [required: >=1.10.4, installed: 1.16.0]
pip-tools==7.3.0
├── build [required: Any, installed: 1.0.3]
│ ├── colorama [required: Any, installed: 0.4.6]
│ ├── packaging [required: >=19.0, installed: 23.2]
│ ├── pyproject-hooks [required: Any, installed: 1.0.0]
│ │ └── tomli [required: >=1.1.0, installed: 2.0.1]
│ └── tomli [required: >=1.1.0, installed: 2.0.1]
├── click [required: >=8, installed: 8.1.7]
│ └── colorama [required: Any, installed: 0.4.6]
├── pip [required: >=22.2, installed: 24.0]
├── setuptools [required: Any, installed: 69.1.1]
├── tomli [required: Any, installed: 2.0.1]
└── wheel [required: Any, installed: 0.42.0]
pipdeptree==2.13.2
pipenv==2023.11.17
├── certifi [required: Any, installed: 2022.12.7]
├── setuptools [required: >=67, installed: 69.1.1]
└── virtualenv [required: >=20.24.2, installed: 20.25.0]
├── distlib [required: >=0.3.7,<1, installed: 0.3.8]
├── filelock [required: >=3.12.2,<4, installed: 3.13.1]
└── platformdirs [required: >=3.9.1,<5, installed: 4.1.0]
pipgrip==0.10.12
├── anytree [required: >=2.4.1, installed: 2.12.1]
│ └── six [required: Any, installed: 1.16.0]
├── click [required: >=7, installed: 8.1.7]
│ └── colorama [required: Any, installed: 0.4.6]
├── packaging [required: >=17, installed: 23.2]
├── pip [required: >=22.2, installed: 24.0]
├── setuptools [required: >=38.3, installed: 69.1.1]
└── wheel [required: Any, installed: 0.42.0]
pyclibrary==0.2.2
└── pyparsing [required: >=2.3.1,<4, installed: 3.1.1]
pycuda==2024.1
├── appdirs [required: >=1.4.0, installed: 1.4.4]
├── Mako [required: Any, installed: 1.3.0]
│ └── MarkupSafe [required: >=0.9.2, installed: 2.1.3]
└── pytools [required: >=2011.2, installed: 2023.1.1]
├── platformdirs [required: >=2.2.0, installed: 4.1.0]
└── typing-extensions [required: >=4.0, installed: 4.9.0]
pytest-benchmark==4.0.0
├── py-cpuinfo [required: Any, installed: 9.0.0]
└── pytest [required: >=3.8, installed: 8.0.1]
├── colorama [required: Any, installed: 0.4.6]
├── exceptiongroup [required: >=1.0.0rc8, installed: 1.2.0]
├── iniconfig [required: Any, installed: 2.0.0]
├── packaging [required: Any, installed: 23.2]
├── pluggy [required: >=1.3.0,<2.0, installed: 1.4.0]
└── tomli [required: >=1.0.0, installed: 2.0.1]
requests==2.28.1
├── certifi [required: >=2017.4.17, installed: 2022.12.7]
├── charset-normalizer [required: >=2,<3, installed: 2.1.1]
├── idna [required: >=2.5,<4, installed: 3.4]
└── urllib3 [required: >=1.21.1,<1.27, installed: 1.26.13]
rich==13.7.0
├── markdown-it-py [required: >=2.2.0, installed: 3.0.0]
│ └── mdurl [required: ~=0.1, installed: 0.1.2]
└── pygments [required: >=2.13.0,<3.0.0, installed: 2.17.2]
tensorrt==8.6.1
tensorrt-dispatch==8.6.1
tensorrt-lean==8.6.1
torchaudio==2.2.1+cu121
└── torch [required: ==2.2.1+cu121, installed: 2.2.1+cu121]
├── filelock [required: Any, installed: 3.13.1]
├── fsspec [required: Any, installed: 2023.4.0]
├── Jinja2 [required: Any, installed: 3.1.2]
│ └── MarkupSafe [required: >=2.0, installed: 2.1.3]
├── networkx [required: Any, installed: 3.0]
├── sympy [required: Any, installed: 1.12]
│ └── mpmath [required: >=0.19, installed: 1.3.0]
└── typing-extensions [required: >=4.8.0, installed: 4.9.0]
torchvision==0.17.1+cu121
├── numpy [required: Any, installed: 1.26.3]
├── Pillow [required: >=5.3.0,!=8.3.*, installed: 9.3.0]
└── torch [required: ==2.2.1+cu121, installed: 2.2.1+cu121]
├── filelock [required: Any, installed: 3.13.1]
├── fsspec [required: Any, installed: 2023.4.0]
├── Jinja2 [required: Any, installed: 3.1.2]
│ └── MarkupSafe [required: >=2.0, installed: 2.1.3]
├── networkx [required: Any, installed: 3.0]
├── sympy [required: Any, installed: 1.12]
│ └── mpmath [required: >=0.19, installed: 1.3.0]
└── typing-extensions [required: >=4.8.0, installed: 4.9.0]
uff==0.6.9
├── numpy [required: >=1.11.0, installed: 1.26.3]
└── protobuf [required: >=3.3.0, installed: 3.20.3]

[PyroGenesis]

@RGX650 Sorry, I wasn't clear. By "does not go through dependency resolution" I meant resolution of the pytorch package / wheel itself, not of its dependencies.

If you try running poetry add --source torch-cu121 torch (with the source torch-cu121 setup beforehand), you will see that poetry will try to download every torch wheel with the version 2.2.2 so that it can compute the hashes if there are none present. That is a pretty big download.

Also some users are trying to make use of groups and / or extras to have both the CPU and GPU versions of pytorch detailed in their pyproject.toml, the idea being that they could resolve to one or the other based on parameters of the poetry add command run.

But once you have pytorch package resolved, it's dependencies are relatively smaller and faster to resolve, with or without hashes. So by using a pytorch wheel file directly, you aren't really resolving the pytorch package dynamically, and anyone using your setup will need to have the pytorch wheel file present at the exact path you defined in your pyproject.toml file.

[RGX650]

@PyroGenesis Thank you for your time and explainations :)
Just to put a context:
PS: I am not a developper actually, but so don't hesitate to correct me or explain me things :)
I managed to get things work (i guess) with quite some time by trial and errors...
I am on windows, and i had troubles regarding triton and bitsandbytes...among other things...

I don't do poetry add actually.
I came up with making a setup.ps1 with this method:

• set "POETRY_SOLVER_LAZY_WHEEL" = "true"
• set "PIP_EXTRA_INDEX_URL" = "http://localhost:8080/simple https://download.pytorch.org"
• set "PIP_TRUSTED_HOST" = "localhost:8080 download.pytorch.org"
• set "POETRY_SOLVER_LAZY_WHEEL" = "true"
• set "POETRY_INSTALLER_MODERN_INSTALLATION" = "true"
• using poetry config and poetry add source
• Edit pyproject.toml with powershell script directly

Then

• poetry lock --no-update
• poetry debug resolve --tree -vvvv
• poetry lock --no-update
• poetry install --no-root --only main

This way, i skipped using poetry add.
If i remember well, I didn't see poetry download all the wheels and the whole process was actually pretty fast.

I also previously tried different solutions with groups and/or extras in different manners, but it didn't work, and when it did, it seamed like a pretty mess.

[simjak]

This is working for all platforms ( I need only the CPU, but you can add other logic)

[tool.poetry.dependencies]
torch = [
    { version = "^2.2.0", source = "pypi", markers = "sys_platform == 'darwin'"},
    { version = "^2.2.0", source = "pypi", markers = "platform_machine == 'aarch64' and sys_platform != 'darwin'"},
    { version = "^2.2.0", source = "pytorch_cpu", markers = "platform_machine == 'x86_64' and sys_platform != 'darwin'" },
]
torchvision = [
    { version = "^0.17.0", source = "pypi", markers = "sys_platform == 'darwin'"},
    { version = "^0.17.0", source = "pypi", markers = "platform_machine == 'aarch64' and sys_platform != 'darwin'"},
    { version = "^0.17.0", source = "pytorch_cpu", markers = "platform_machine == 'x86_64' and sys_platform != 'darwin'" },
]

[[tool.poetry.source]]
name = "pytorch_cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[ClaytonJY]

@simjak that's great! a bit verbose, but probably the best we can do today.

I think that does exactly what we want as long as your GPU machines and non-GPU machines are different platforms, like a macbook (no GPU) and a linux VM (with GPU). But if you develop on say two different amd64 linux machines, one with a GPU and one without, this will still have you pulling in many gigabytes of nvidia stuff on your non-GPU machine. Your code should still run on both machines, just a bit of unfortunate bloat on the non-GPU machine.

The big painpoint left for me is CI, where I might be using non-GPU runners and want to install all my dependencies, but don't need CUDA to run tests, formatters, linters, etc. Lots of wasted time pulling in the CUDA stuff on every run.