`--extra-pip-args` doesn't pick up `--platform` option for `pip` correclty. #6001

ChannyClaus · 2023-11-01T04:06:06Z

Issue description

--platform flag (along with --no-deps and --target) from pip doesn't seem to get picked up correctly by pipenv via --extra-pip-args.

Expected result

The installation should succeed since pip install does with those flags:

$ pip install --platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/ tensorflow~=2.8.0
Collecting tensorflow~=2.8.0
  Using cached tensorflow-2.8.4-cp38-cp38-macosx_10_14_x86_64.whl (217.9 MB)
Installing collected packages: tensorflow
Successfully installed tensorflow-2.8.4
WARNING: Target directory /Users/chan.kang/.local/share/virtualenvs/tensorflow-4pbUTlUx/lib/python3.8/site-packages/tensorflow-2.8.4.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/chan.kang/.local/share/virtualenvs/tensorflow-4pbUTlUx/lib/python3.8/site-packages/tensorflow already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/chan.kang/.local/share/virtualenvs/tensorflow-4pbUTlUx/lib/python3.8/site-packages/bin already exists. Specify --upgrade to force replacement.

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: python3.8 -m pip install --upgrade pip

Actual result

$ pipenv install --verbose --extra-pip-args "--platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/"  tensorflow~=2.8.0
Installing tensorflow~=2.8.0...
Resolving tensorflow~=2.8.0...
Added tensorflow to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock (db4242) out of date, updating to (904453)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.star
ting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.addi
ng_requirement(SpecifierRequirement('tensorflow~=2.8.0'), None)
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not 
find a version that satisfies the requirement tensorflow~=2.8.0 (from 
versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1)
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pipenv.patched.pip._vendor.resolvelib.resolvers.RequirementsConflicted: 
Requirements conflict: SpecifierRequirement('tensorflow~=2.8.0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_internal/resolutio
n/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pipenv.patched.pip._vendor.resolvelib.resolvers.ResolutionImpossible: 
[RequirementInformation(requirement=SpecifierRequirement('tensorflow~=2.8.0'),
parent=None)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 440, 
in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_internal/resolutio
n/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pipenv.patched.pip._internal.exceptions.DistributionNotFound: No matching 
distribution found for tensorflow~=2.8.0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 675, in 
<module>
    main()
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 661, in main
    _main(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 645, in 
_main
    resolve_packages(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 612, in 
resolve_packages
    results, resolver = resolve(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 592, in 
resolve
    return resolve_deps(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 908, 
in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 681, 
in actually_resolve_deps
    resolver.resolve()
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 442, 
in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mNo matching 
distribution found for tensorflow~=2.8.0[0m
✘ Locking Failed!
⠧ Locking...
Traceback (most recent call last):
  File "/Users/chan.kang/.pyenv/versions/3.8.17/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/install.py", line 297, in do_install
    raise e
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/install.py", line 281, in do_install
    do_init(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/install.py", line 648, in do_init
    do_lock(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/lock.py", line 65, in do_lock
    venv_resolve_deps(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 849, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 718, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

When possible, provide the verbose output (--verbose), especially for locking and dependencies resolving issues.

Steps to replicate

Running the commands in the above snippet in an empty directory is sufficient to reproduce this issue.

$ pipenv --support

Pipenv version: '2023.10.24'

Pipenv location: '/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv'

Python location: '/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8'

OS Name: 'posix'

User pip version: '23.3.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.17',
 'os_name': 'posix',
 'platform_machine': 'arm64',
 'platform_python_implementation': 'CPython',
 'platform_release': '22.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 22.6.0: Wed Jul  5 22:21:53 PDT '
                     '2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6020',
 'python_full_version': '3.8.17',
 'python_version': '3.8',
 'sys_platform': 'darwin'}

System environment variables:

SHELL
LSCOLORS
PYENV_HOOK_PATH
GITLAB_PYPI_USERNAME
PYENV_SHELL
BOKEH_SIGN_SESSIONS
XPC_FLAGS
NVM_INC
TERM_PROGRAM_VERSION
REACT_APP_RECAPTCHA_SITE_KEY
GITLAB_PYPI_PROJECT_ID
__CFBundleIdentifier
SSH_AUTH_SOCK
TERM_SESSION_ID
HOMEBREW_PREFIX
SF_WAREHOUSE
GITLAB_PYPI_PASSWORD
PYENV_VERSION
PWD
LOGNAME
STAGE
LLVM_CONFIG
SF_USERNAME
MANPATH
RDS_PASSWORD
LaunchInstanceID
LDFLAGS
HOME
RDS_DB_NAME
LANG
SECURITYSESSIONID
UNLEASH_INSTANCE_ID
SF_ROLE
TMPDIR
CPPFLAGS
CLICOLOR
LOCAL_GRAPHQL_DEV
SF_DATABASE
PYENV_DIR
UNLEASH_URL
INFOPATH
RDS_PORT
NVM_DIR
RDS_HOSTNAME
TERM
SF_ACCOUNT
GITLAB_READ_PACKAGE_REGISTRY_TOKEN
USER
HOMEBREW_CELLAR
RDS_USERNAME
SHLVL
NVM_CD_FLAGS
HOMEBREW_REPOSITORY
SF_PASSWORD
XPC_SERVICE_NAME
PYENV_ROOT
SF_SCHEMA
PATH
BOKEH_SECRET_KEY
NVM_BIN
RECAPTCHA_SECRET_KEY
BOKEH_PY_LOG_LEVEL
OLDPWD
TERM_PROGRAM
__CF_USER_TEXT_ENCODING
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /Users/chan.kang/.pyenv/versions/3.8.17/bin:/Users/chan.kang/.pyenv/versions/3.8.17/bin:/opt/homebrew/Cellar/pyenv/2.3.23/libexec:/opt/homebrew/Cellar/pyenv/2.3.23/plugins/python-build/bin:/Users/chan.kang/.nvm/versions/node/v18.16.1/bin:/Users/chan.kang/.pyenv/shims:/opt/homebrew/opt/postgresql@15/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/findutils/libexec/gnubin:/opt/homebrew/opt/libpq/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Users/chan.kang/.cargo/bin:/Users/chan.kang/.gem/ruby/X.X.0/bin:/usr/local/opt/openjdk/bin:/Users/chan.kang/.local/bin:/opt/homebrew/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/local/opt/grep/libexec/gnubin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
SHELL: /opt/homebrew/Cellar/bash/5.2.15/bin/bash
LANG: en_US.UTF-8
PWD: /Users/chan.kang/test/tensorflow2

Contents of Pipfile ('/Users/chan.kang/test/tensorflow2/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.8"

The text was updated successfully, but these errors were encountered:

ChannyClaus · 2023-11-02T04:05:00Z

took a quick look at the code - seems like this flag isn't passed down to the locking function https://github.com/pypa/pipenv/blob/main/pipenv/routines/install.py#L648-L655

is pipenv not designed to handle the case where the user is trying to masquerade a platform different from the one they are running pipenv on? (i'm trying to create a lock for Pipfile where the dependencies exist only for x86-64, i.e. not for arm64)

matteius · 2023-11-05T14:11:06Z

@ChannyClaus I opened a PR that I believe addresses it. #6006
I don't think you'll need to pass --no-deps since we already pass this as part of pipenv installation.

ChannyClaus · 2023-11-05T15:12:47Z

@ChannyClaus I opened a PR that I believe addresses it. #6006 I don't think you'll need to pass --no-deps since we already pass this as part of pipenv installation.

hmm does the command above pipenv install --verbose --extra-pip-args "--platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/" tensorflow~=2.8.0 work with the change? i just tried it out locally and it seems to be failing still.

matteius · 2023-11-05T15:24:42Z

Well I wouldn't pass --no-deps because as I said we already pass that by default, but I wasn't able to get it to work with the specifier ~=2.8.0 but removing that it did work.

ChannyClaus · 2023-11-05T15:30:39Z

yeah i think making it work with the specifier is critical - at least for the use case we have a bunch of apple silicon macbook, i.e.

In [1]: import platform

In [2]: platform.machine()
Out[2]: 'arm64'

users working with a Pipfile that pins tensorflow at ~=2.8.0. not exactly sure how pipenv lock works under the hood but if it's using pip under the hood it supposedly should work since pip install tensorflow~=2.8.0 works with the flags in the PR description.

matteius · 2023-11-05T15:33:21Z

Well something is wrong with that version for it to be resolved, version specifiers in generally are working.

matteius · 2023-11-05T15:35:37Z

Actually my reason it wasn't resolving is I have default python 3.11 and that version only goes up to 3.11 -- when I run this it works:

$ pipenv install --verbose --extra-pip-args "--platform=win_amd64 --target C:\\c\\Users\\matte\\.virtualenvs\\issue-6001-WKuyds7q\\Lib\\site-packages\\"  tensorflow~=2.8.0 --python 3.10

matteius · 2023-11-05T15:36:28Z

The other thing my PR does not address is how to store extra pip args in the Pipfile for future relocking so it doesn't have to be supplied on the CLI everytime. Also I am pretty sure in my PR i haven't exposed it directly to the lock command yet but that should be easy enough to add.

ChannyClaus · 2023-11-05T15:48:30Z

$ git fetch
g(pipenv) chan.kang@Chans-MBP-743 ~/repos/pipenv -  (issue-6001-extra-pip-args)
$ git reset --hard origin/issue-6001-extra-pip-args
HEAD is now at 12278956 Use the extra pip args in the resolver.
(pipenv) chan.kang@Chans-MBP-743 ~/repos/pipenv -  (issue-6001-extra-pip-args)
$  pipenv install --verbose --extra-pip-args "--platform=win_amd64 --target test"  tensorflow~=2.8.0 --python 3.10
Installing tensorflow~=2.8.0...
Resolving tensorflow~=2.8.0...
Added tensorflow to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock (f2a69e) out of date, updating to (5bfae3)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('pytz'), None)
INFO:pipenv.patched.pip._internal.operations.prepare:Collecting pytz (from -r 
/var/folders/s4/dtm89b1x7zxcy0ly0frk041c0000gq/T/pipenv-e2w65hf3-requirements/pipenv-ydqbue5q-constraints.txt (line 4))
INFO:pipenv.patched.pip._internal.network.download:Using cached pytz-2023.3.post1-py2.py3-none-any.whl.metadata (22 kB)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('tensorflow~=2.8.0'
), None)
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement 
tensorflow~=2.8.0 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1)
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pipenv.patched.pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: 
SpecifierRequirement('tensorflow~=2.8.0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pipenv.patched.pip._vendor.resolvelib.resolvers.ResolutionImpossible: 
[RequirementInformation(requirement=SpecifierRequirement('tensorflow~=2.8.0'), parent=None)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 444, in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pipenv.patched.pip._internal.exceptions.DistributionNotFound: No matching distribution found for tensorflow~=2.8.0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 675, in <module>
    main()
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 661, in main
    _main(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 645, in _main
    resolve_packages(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 612, in resolve_packages
    results, resolver = resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 592, in resolve
    return resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 917, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 685, in actually_resolve_deps
    resolver.resolve()
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 446, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mNo matching distribution found for tensorflow~=2.8.0[0m
✘ Locking Failed!
⠧ Locking packages...
Traceback (most recent call last):
  File "/Users/chan.kang/.local/share/virtualenvs/pipenv-Ex9cqLYG/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 297, in do_install
    raise e
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 281, in do_install
    do_init(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 648, in do_init
    do_lock(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/lock.py", line 66, in do_lock
    venv_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 858, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 722, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

hmm something still seems awry on my end (just tried the command that worked for you); maybe i'm doing something wrong? (it seems to be using the correct code based on the stacktrace)

matteius · 2023-11-05T15:51:11Z

What other packages in your Pipfile dependencies might be affecting your resolution? maybe something is requiring a newer version of tensorflow.

matteius · 2023-11-05T15:51:29Z

Well wait, win_amd64 isn't going to work for you on a MBP ...

matteius · 2023-11-05T15:53:03Z

Looks like the wheel you are targeting is [macosx_10_14_x86_64] as you had in your original comment (https://files.pythonhosted.org/packages/b1/c7/0df8a0819e16af0fdd38791dc048af621a089fb952e313206bb9e6fea42d/tensorflow-2.8.3-cp310-cp310-macosx_10_14_x86_64.whl)

ChannyClaus · 2023-11-05T17:20:25Z

Well wait, win_amd64 isn't going to work for you on a MBP ...

well that was actually kinda why i created this issue - to be able to feed platforms different from my local when installing / locking (our use case is for masquarading x86-64 but the implementation should work for all platforms, i suspect?). for example, pip install works:

$ pip install tensorflow~=2.8.0 --platform=win_amd64 --target test --no-deps
Collecting tensorflow~=2.8.0
  Downloading tensorflow-2.8.4-cp310-cp310-win_amd64.whl (438.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 438.4/438.4 MB 9.6 MB/s eta 0:00:00
Installing collected packages: tensorflow
Successfully installed tensorflow-2.8.4

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
chan.kang@Chans-MBP-743 ~/repos/pipenv -  (issue-6001-extra-pip-args)
$

and i was hoping we'd be able to get pipenv to work with it as well.

matteius · 2023-11-05T17:32:33Z

Well you haven't answered my question -- do you have anything else in your Pipfile being specified? I showed an example of it working with my branch, but I had to downgrade from python 3.11 -- however your logs indicate something else in the Pipfile is restricting the resolver to a version of tensorflow higher than what you are installing. Installing it as a 1-off with pip isn't the same check if you have anything else defined in your Pipfile.

ChannyClaus · 2023-11-05T17:56:10Z

sorry about that. fails the same way in a newly created dir:

$ rm -rf new_dir && mkdir new_dir && cd new_dir && pipenv install --verbose --extra-pip-args "--platform=win_amd64 --target test"  tensorflow~=2.8.0 --python 3.10
Creating a Pipfile for this project...
Installing tensorflow~=2.8.0...
Resolving tensorflow~=2.8.0...
Added tensorflow to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('tensorflow~=2.8.0
'), None)
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement 
tensorflow~=2.8.0 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.15.0rc0, 
2.15.0rc1)
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pipenv.patched.pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: 
SpecifierRequirement('tensorflow~=2.8.0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pipenv.patched.pip._vendor.resolvelib.resolvers.ResolutionImpossible: 
[RequirementInformation(requirement=SpecifierRequirement('tensorflow~=2.8.0'), parent=None)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 444, in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pipenv.patched.pip._internal.exceptions.DistributionNotFound: No matching distribution found for tensorflow~=2.8.0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 675, in <module>
    main()
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 661, in main
    _main(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 645, in _main
    resolve_packages(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 612, in resolve_packages
    results, resolver = resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 592, in resolve
    return resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 917, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 685, in actually_resolve_deps
    resolver.resolve()
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 446, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mNo matching distribution found for tensorflow~=2.8.0[0m
✘ Locking Failed!
⠸ Locking packages...
Traceback (most recent call last):
  File "/Users/chan.kang/.local/share/virtualenvs/pipenv-Ex9cqLYG/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 297, in do_install
    raise e
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 281, in do_install
    do_init(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 672, in do_init
    do_lock(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/lock.py", line 66, in do_lock
    venv_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 858, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 722, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!
(pipenv) chan.kang@Chans-MBP-755 ~/new_dir - 
$

matteius · 2023-11-08T02:40:24Z

@ChannyClaus you are still specifying a windows platform in that example. Digging a bit deeper, you have a arm64 architecture, but the 2.8 wheels do not have a mac os arm wheel (just x86 as tensorflow-2.8.0-cp310-cp310-macosx_10_14_x86_64.whl or tensorflow-2.8.4-cp310-cp310-macosx_10_14_x86_64.whl is the latest in the 1.8.x chain which is what you are specifying). Even in your example pip is installing the x86 whl, which doesn't seem right for your system (but is what you are specifying). Since there are only prebuilt wheels for this package, no sdist will be built and the pipenv resolver fails.

I am not quite sure why it wouldn't fail with the pip example, but we see that in the stack trace that there are newer wheels that match your system, specifically: CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement tensorflow~=2.8.0 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.15.0rc0, 2.15.0rc1) which would be the macosx_12_0_arm64.whl.

FWIW, you should be able to simplify the command you are running:
pip install --platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/ tensorflow~=2.8.0
to pipenv run pip install --platform=macosx_10_14_x86_64 --no-deps tensorflow~=2.8.0 but it sill wouldn't be in the lockfile or Pipfile.

matteius added the triage label Nov 1, 2023

matteius linked a pull request Nov 5, 2023 that will close this issue

Supply the extra pip args in the resolver. #6006

Open

2 tasks

matteius self-assigned this Nov 5, 2023

matteius added PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. Type: Enhancement 💡 This is a feature or enhancement request. labels Nov 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`--extra-pip-args` doesn't pick up `--platform` option for `pip` correclty. #6001

`--extra-pip-args` doesn't pick up `--platform` option for `pip` correclty. #6001

ChannyClaus commented Nov 1, 2023 •

edited

ChannyClaus commented Nov 2, 2023 •

edited

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023 •

edited

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023 •

edited

ChannyClaus commented Nov 5, 2023 •

edited

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023 •

edited

matteius commented Nov 8, 2023

--extra-pip-args doesn't pick up --platform option for pip correclty. #6001

--extra-pip-args doesn't pick up --platform option for pip correclty. #6001

Comments

ChannyClaus commented Nov 1, 2023 • edited

Issue description

Expected result

Actual result

Steps to replicate

ChannyClaus commented Nov 2, 2023 • edited

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023 • edited

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023

matteius commented Nov 5, 2023 • edited

ChannyClaus commented Nov 5, 2023 • edited

matteius commented Nov 5, 2023

ChannyClaus commented Nov 5, 2023 • edited

matteius commented Nov 8, 2023

`--extra-pip-args` doesn't pick up `--platform` option for `pip` correclty. #6001

`--extra-pip-args` doesn't pick up `--platform` option for `pip` correclty. #6001

ChannyClaus commented Nov 1, 2023 •

edited

ChannyClaus commented Nov 2, 2023 •

edited

ChannyClaus commented Nov 5, 2023 •

edited

matteius commented Nov 5, 2023 •

edited

ChannyClaus commented Nov 5, 2023 •

edited

ChannyClaus commented Nov 5, 2023 •

edited