Poetry cannot install package with inconsistent wheel metadata

[arthi-navenio]

• Poetry version: Poetry (version 1.2.1)

• Python version: Python 3.9.9

• OS version and name: Ubuntu 20.04.5 LTS (Focal Fossa)

• pyproject.toml: https://gist.github.com/arthi-navenio/df7283f9ce613b8c2df563b1a5972736

• I am on the latest stable Poetry version, installed using a recommended method.

• I have searched the issues of this repo and believe that this is not a duplicate.

• I have consulted the FAQ and blog for any relevant entries or release notes.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Package hosted in private aritifactory not discovered by poetry when performing an installation for specific python version. Packages in private aritifactory are hosted in the format shown below

- pypi
        - custom-package
        - 20220916.9
                         - <wheel_compatible_with_python3.10>
                         - <wheel_compatible_with_python3.9>
                         - <wheel_compatible_with_python3.8>

pip install custom-package==20220916.9 automatically discovers and installs relevant package from private artifactory based on the compatible python version. However, doing the same with poetry raises the error

Because depends on custom-package (20220916.9) which doesn't match any versions, version solving failed.

Complete log - https://gist.github.com/arthi-navenio/d0ee9cbe8e36c949a8329a33cbc6fcce

I tried revising the pyproject.toml file to specify exact url like the below for installing the custom package

custom-package = { url = "https://custom.jfrog.io/artifactory/api/pypi/pypi/simple/custom-package/custom-package-20220916.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl" }

and that fails with

404 Client Error: Not Found for url: https://custom.jfrog.io/artifactory/api/pypi/pypi/simplecustom-package/custom-package-20220916.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl

Complete log - https://gist.github.com/arthi-navenio/8cf051bb5cced1fdb699111df59fbc52

even though the credentials are added with poetry config http-basic <repo-name-as-seen-in-source-pyproject.toml> user pass

I have spent time exhaustively searching for another alternative and seems like a bug with poetry.

[neersighted]

Could you please try re-locking first? poetry install will just use the lock file, and since you haven't re-locked Poetry hasn't actually tried to find the files yet.

[arthi-navenio]

Sure. Performed poetry update

Updating dependencies
Resolving dependencies... (9.5s)

The current project's Python requirement (3.9.9) is not compatible with some of the required packages Python requirement:
  - custom-package requires Python >=3.10, <3.11, so it will not be satisfied for Python 3.9.9

Because xxx depends on custom-package (20220916.9) which requires Python >=3.10, <3.11, version solving failed.

  • Check your dependencies Python requirement: The Python requirement can be specified via the python or markers properties
    
    For custom-package, a possible solution would be to set the python property to "<empty>"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

Performing update raises the above error when clearly there is a python package in that directory for python3.9

[neersighted]

Ah, we might be making some progress -- you need to inspect the constraints/metadata of your custom-package -- if it claims >=3.10, <3.11, it doesn't matter if you built a Python 3.9 wheel -- the 3.9 wheel claims that it doesn't run on 3.9!

Poetry is telling you exactly what is happening here -- I'm going to close this for now, but we can convert it to a discussion if you need support, or you can join Discord where you can talk with a maintainer and share private details with them if you need more help getting to the bottom of your constraints.

[arthi-navenio]

I am happy to do either. The 3 wheels inside that directory have 3 separate python version expectations. Whilst pip does not seem to be having a problem in detecting and installing it, poetry fails

[neersighted]

The metadata between all your wheels needs to be the same -- that is the current origin of your issues then. Currently Poetry will try to pick the first wheel and hope metadata is consistent if there is not a sdist available.

This changed in #6547 -- however that PR is scoped to 1.3 only and will not help in this case.

Poetry does not support packages with inconsistent metadata between artifacts (and you shouldn't be setting Requires-Python differently -- the package supports all Python versions in the full range, it's just the individual artifacts that do not, and they capture that in the wheel tags) -- fix that and everything should just work.

[arthi-navenio]

Thank you, I will check with my team internally.