Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

virtualenvs.prefer-active-python uses poetry's Python venv, not the active pyenv virtualenv #8119

Open
4 tasks done
paultiplady opened this issue Jun 21, 2023 · 3 comments · May be fixed by #8120
Open
4 tasks done
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged

Comments

@paultiplady
Copy link

paultiplady commented Jun 21, 2023

  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

The (experimental) virtualenvs.prefer-active-python flag doesn't seem to be working for me.

I set up a local pyenv venv with:

pyenv virtualenv 3.10.10 pet
pyenv local pet

And this venv is activated automatically in my shell. However Poetry isn't picking it up; if I say

> poetry env info -vvv
Loading configuration file /Users/paul/Library/Application Support/pypoetry/config.toml

Virtualenv
Python:         3.10.10
Implementation: CPython
Path:           NA
Executable:     NA

System
Platform:   darwin
OS:         posix
Python:     3.10.10
Path:       /Users/paul/.pyenv/versions/3.10.10
Executable: /Users/paul/.pyenv/versions/3.10.10/bin/python3.10

Reading the code, I don't get how this flag is supposed to work; tracing the local code execution we correctly run:

prefer_active_python = self._poetry.config.get(
"virtualenvs.prefer-active-python"
)
python_minor = self.get_python_version(
precision=2, prefer_active_python=prefer_active_python, io=self._io
).to_string()

And determine the minor version of the local python to be 3.10. Then we run

if not create_venv:
return self.get_system_env()

And fetch the system python (which is /Users/paul/.pyenv/versions/3.10.10/bin/python3.10 in this case, the pyenv system version that I installed poetry with).

I can't see how this code is supposed to work; it's just extracting python_minor from the pyenv interpreter, and maybe looking that up in the poetry virtualenv repo? But of course that won't work for venvs managed by pyenv. It seems to me that EnvManager.get() should just say:

        if prefer_active_python:
            executable = self._detect_active_python()
            bin_dir = executable.parent.parent
            path = bin_dir.parent
            return VirtualEnv(path, Path(executable.name))

Hacking that change in gives me this output:

Virtualenv
Python:         3.10.10
Implementation: CPython
Path:           /Users/paul/.pyenv/versions/pet
Executable:     /Users/paul/.pyenv/versions/pet/bin/python
Valid:          True

System
Platform:   darwin
OS:         posix
Python:     3.10.10
Path:       /Users/paul/.pyenv/versions/3.10.10
Executable: /Users/paul/.pyenv/versions/3.10.10/bin/python3.10

This seems to be the right interpreter. Does this seem like the right path?

@paultiplady paultiplady added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Jun 21, 2023
paultiplady added a commit to paultiplady/poetry that referenced this issue Jun 21, 2023
The virtualenvs.prefer-active-python flag is currently not
working for me; instead of selecting the local env, I get they
poetry venv.

This change resolves the problem for me.

FIXES: python-poetry#8119
@paultiplady paultiplady linked a pull request Jun 21, 2023 that will close this issue
2 tasks
@finswimmer
Copy link
Member

Hey @paultiplady,

I'm not sure I can follow you. Maybe you misunderstand what virtualenvs.prefer-active-python is doing? It's saying: "If Poetry creates a venv it should pick up the current activated python interpreter to create this venv".

So can you please point out what you've expected to see and what happens instead? This makes it easier to understand whether there is an issue or not.

fin swimmer

@paultiplady
Copy link
Author

paultiplady commented Jun 21, 2023

Thanks, it looks like I misunderstood what's going on with this flag then. I read #4852 and perhaps didn't read too much into "the pyenv way", assuming that meant how I was familiar with folks using Pyenv.

The workflow that I used with Poetry from when it first came out until when it broke some time in the last year was to use pyenv with the pyenv-virtualenv plugin to manage the venv. (The pyenv incantations I shared at the top of my original post). This means when I cd into the project directory, my venv is automatically activated, and I don't need to say poetry run <python-entrypoint> or poetry shell to use in the venv, I just have my venv activated automatically. (I'd previously just install poetry inside the per-project venv, and that worked fine up until some point between 2022 and now.)

In my Dockerfiles I was using virtualenvs.create=False but I don't think I was explicitly setting that in the dev env config. (I do have that flag set now on my new "official installer" install.)

It feels like the code that currently exists is pretty close to working as I expect it with virtualenvs.prefer-active-python=True & virtualenvs.create=False, and regardless of whether that's intended to be supported, I suspect there is still a bug; with prefer-active-python=True shouldn't poetry env info show the currently-active python (in this case a virtualenv), not the system python that was used to install poetry?

To be clear, in the diags I provided above, this command:

> poetry env info -vvv
Loading configuration file /Users/paul/Library/Application Support/pypoetry/config.toml

Virtualenv
Python:         3.10.10
Implementation: CPython
Path:           NA
Executable:     NA

System
Platform:   darwin
OS:         posix
Python:     3.10.10
Path:       /Users/paul/.pyenv/versions/3.10.10
Executable: /Users/paul/.pyenv/versions/3.10.10/bin/python3.10

Is being run inside the venv /Users/paul/.pyenv/versions/pet/bin/python. I want poetry to use the active python (in this case a venv), not infer the system python.

For what it's worth, I retested this using a non-venv pyenv version and the instructions in https://python-poetry.org/docs/managing-environments/, and that's broken in the same way; I think that is the expected usecase from the docs:

❯ pyenv local 3.9.9

❯ poetry env info

Virtualenv
Python:         3.10.10
Implementation: CPython
Path:           NA
Executable:     NA

System
Platform:   darwin
OS:         posix
Python:     3.10.10
Path:       /Users/paul/.pyenv/versions/3.10.10
Executable: /Users/paul/.pyenv/versions/3.10.10/bin/python3.10

In this case we should be pulling the 3.9 pyenv version as the active python version, and instead it's pulling 3.10.10 (the pyenv system version, used to install poetry).

My fix resolves this case too (though I see now that we can't hardcode the executable to a VirtualEnv, since it might be a non-venv interpreter. Maybe it can be a GenericEnv?)

@jonvet
Copy link

jonvet commented Jan 4, 2024

I also misunderstood how virtualenvs.prefer-active-python = true should work. however I just wasn't able to use pyenv & poetry together: I had the same problem that poetry install would create a new venv, even though I had virtualenvs.prefer-active-python = true and my pyenv venv was active.

with virtualenvs.create = false it would install everything into the system python (a pyenv version in my case). using @paultiplady's fix it works for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants