Fetching package info on non-pypa link

[Abhishek-Dev09-zz]

Environment Versions

1. OS Type
2. Python version: 3.6.8
3. pip version: 20.1.1
4. pip-tools version: 5.2.1

Steps to replicate

pip-compile --find-links=thirdparty --generate-hashes --index-url=https://github.com/nexB/scancode-toolkit/tree/develop/thirdparty

Actual result

Fetch package info from PyPI failed: https://github.com/nexB/scancode-toolkit/tree/develop/pypi/soupsieve/json: HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /nexB/scancode-toolkit/tree/develop/pypi/soupsieve/json (Caused by ResponseError('too many 500 error responses',)) Couldn't get hashes from PyPI, fallback to hashing files Hashing soupsieve-1.9.2-py2.py3-none-any.whl pygments

Expected result - it would directly generate hash from file. For example-

Hashing soupsieve-1.9.2-py2.py3-none-any.whl pygments

[atugushev]

Hello @Abhishek-Dev09,

Thanks for the issue! That should work:

$ git clone https://github.com/nexB/scancode-toolkit
$ pip-compile --generate-hashes --find-links=scancode-toolkit/thirdparty 

[atugushev]

You cannot pass GitHub tree URL to --index-url, see the doc:

Base URL of the Python Package Index (default https://pypi.org/simple). This should point to a repository compliant with PEP 503 (the simple repository API) or a local directory laid out in the same format.

[Abhishek-Dev09-zz]

Hello @Abhishek-Dev09,

Thanks for the issue! That should work:

$ git clone https://github.com/nexB/scancode-toolkit
$ pip-compile --generate-hashes --find-links=scancode-toolkit/thirdparty 

@atugushev : Thanks, But I do not want hashing from pypi instead it fetches from find-links options because deps of my requirenment.txt is not available on pypi.

[atugushev]

@Abhishek-Dev09

Ah, I see your point. A package, let's say, six from --find-links=./links/ gets its hashes from PyPI, but it should get hashes from a file ./links/six-1.10.tar.gz. in links directory. Thanks!

[Abhishek-Dev09-zz]

@Abhishek-Dev09

Ah, I see your point. A package, let's say, six from --find-links=./links/ gets its hashes from PyPI, but it should get hashes from a file ./links/six-1.10.tar.gz. in links directory. Thanks!

@atugushev : Yes, you understand my point.

[Abhishek-Dev09-zz]

@pombredanne As we are dependent on this tools , soon this bug will be solved , then it get hash directly from wheel in 1st attempt only instead first from pypi then from wheel file in second attempt.

[atugushev]

Hashing packages from find links was fixed in #1723.