pipenv doesn't honor NETRC environment variable

[bigpick]

Issue description

pipenv does not honor the NETRC environment variable.

... a location outside of the home directory can be set using the environment variable NETRC ...

Expected result

pipenv should leverage the path specified in the NETRC environment variable; requests library does this already:

• Original PR to add: Respect the NETRC environment variable psf/requests#5643
• Relevant current (requests) source: here

Actual result

pipenv does not use the set NETRC variable, defaulting to only ~/.netrc and ~/_netrc paths. If using a private source (e.g Artifactory) as a url in [[source]], this results in a [pipenv.exceptions.ResolutionFailure]:

...
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Could not find a version that matches asyncssh (from -r .... (line 3))
No versions found
Was <private artifactory URL> reachable?

Copying the .netrc file from the custom dir to either ~/.netrc and/or ~/_netrc then successfully works with no other modifications to the Pipfile file (and it resolves with public sources fine as well, so is not an issue with the packages/dependencies or something like that)

Steps to replicate

Pipfile:

[[source]]
url = "<some private pypi repository URL>"
verify_ssl = true
name = "Private PyPi"

[packages]
requests = "*"
asyncssh = "*"

[dev-packages]

[requires]
python_version = "3.10"

Create a .netrc in some directory path thats not ~/.netrc or ~/_netrc (and make sure that there are no files there currently, otherwise, temporarily move them to something like ~/.netrc.temp_backup.

machine <private repo base URL>
login <user>
password <pw | key>

Then set NETRC to the path to that file:

export NETRC=/some/path/to/that/.netrc

Then, try:

pipenv install

---

pipenv --version
pipenv, version 2021.5.29

[mmraz]

This functionality is especially useful during automated builds, such as on a Jenkins worker, where the withCredentials file reference gets set into a NETRC environment variable. Overriding the worker execution user's $HOME or overwriting the $HOME/.netrc outside of the build context is very problematic. We've had to use containerized agents which mount a .netrc file at as an agent for stages which call pipenv to work around this.

[bigpick]

I've played around a bit locally trying to figure out where the variable is being lost, since it appears that it should be getting used/checked, as that bit of logic from the requests library is currently included in the vendored requests code, but for some reason, falls through unused

---

My understanding of the pipenv project structure is basically 0, so grain of salt, but the only thing I could find that seemed relevant was that the current notpip/.../requests version doesn't have the recent-enough requests code to leverage the NETRC values

[bigpick]

It looks like there is currently (or, was, hasn't been updated in 8 months) an outstanding WIP draft PR to bump the pip version to 21.0, which includes the required requests version to support NETRC envar.

[bigpick]

bumping pip to 21.0 seemed to have supposedly included as part of Switch the resolver from piptools to pip internal; I'll have to try grabbing latest versions and testing to see if is now honored as expected

[matteius]

@bigpick any updates?

[bigpick]

Hey 👋 , sorry for the delay, but it looks good now!

Checking what I have currently (trying this just now on macOS, fwiw):

pipenv, version 2022.1.8

With a test Pipfile like so:

[[source]]
url = "https://privatehost/artifactory/api/pypi/privaterepo/simple"
verify_ssl = true
name = "Private Repo"

[packages]
requests = "*"
asyncssh = "*"

[dev-packages]

[requires]
python_version = "3.10"

and a ~/.netrc of

machine privatehost
login <user>
password <pass>

A pipenv install works properly, and the Pipfile.lock shows the expected source:

        "sources": [
            {
                "name": "Private Repo",
                "url": "https://privatehost/artifactory/api/pypi/privaterepo/simple",
                "verify_ssl": true
            }
        ]

---

Doing a clean re-install with a custom NETRC envar also works:

rm Pipfile.lock
pipenv --rm

mv ~/.netrc /some/non/default/dir/.netrc
export NETRC=/some/non/default/dir/.netrc

pipenv install

(i.e it also successfully installs and shows the expected source in the lock file)