the simplest 'poetry install' fails #5490

mr-xeroth · 2022-04-25T08:33:31Z

Working environment is Python 3.10.4 on Ubuntu 22.04 LTS

user@comp:~$ poetry config virtualenvs.in-project true
user@comp:~$ poetry new first
Created package first in first
user@comp:~$ cd first
user@comp:~/first$ poetry install
Creating virtualenv first in /home/user/first/.venv
Updating dependencies
Resolving dependencies... (3.6s)

Writing lock file

Package operations: 1 install, 7 updates, 0 removals

  • Updating pyparsing (2.4.7 /usr/lib/python3/dist-packages -> 3.0.8)
  • Updating attrs (21.4.0 /home/user/.local/lib/python3.10/site-packages -> 21.4.0)
  • Updating more-itertools (8.10.0 /usr/lib/python3/dist-packages -> 8.12.0)
  • Updating packaging (21.3 /home/user/.local/lib/python3.10/site-packages -> 21.3)
  • Updating pluggy (1.0.0 /home/user/.local/lib/python3.10/site-packages -> 0.13.1)
  • Updating py (1.11.0 /home/user/.local/lib/python3.10/site-packages -> 1.11.0)
  • Installing wcwidth (0.2.5)
  • Updating pytest (7.1.2 /home/user/.local/lib/python3.10/site-packages -> 5.4.3)

Installing the current project: first (0.1.0)  - Failed to create first.pth for /home/user/first
  - Failed to find a suitable script installation directory for /home/user/first

  OSError

  Unable to access any of "/usr/local/lib/python3.10/dist-packages/first-0.1.0.dist-info"

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:265 in _path_method_wrapper
       261│ 
       262│         if results:
       263│             return results
       264│ 
    →  265│         raise OSError("Unable to access any of {}".format(paths_csv(candidates)))
       266│ 
       267│     def write_text(self, path, *args, **kwargs):  # type: (Path, *Any, **Any) -> Path
       268│         return self._path_method_wrapper(path, "write_text", *args, **kwargs)[0]
       269│

The text was updated successfully, but these errors were encountered:

abn · 2022-04-25T10:31:19Z

@mr-xeroth looks to me for some reason, Poetry thinks the active project environment is the system environment. Can you provide the following?

Poetry version and how it was installed.
Output of poetry config --list.
Output of python3.10 -m site.
Commands to reproduce this in a clean environment, ideally, in the docker.io/ubuntu:22.04 container.

mr-xeroth · 2022-04-25T16:15:18Z

here's my docker log. no issues, gotta dig my stuff on my own.

root@a326e8cb61dd:~# PATH="/root/.local/bin:$PATH"
root@a326e8cb61dd:~# poetry --version
Poetry version 1.1.13
root@a326e8cb61dd:~# cat /etc/issue
Ubuntu 22.04 LTS \n \l

root@a326e8cb61dd:~# poetry config --list
cache-dir = "/root/.cache/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs"  # /root/.cache/pypoetry/virtualenvs
root@a326e8cb61dd:~# python3.10 -m site
sys.path = [
    '/root',
    '/usr/lib/python310.zip',
    '/usr/lib/python3.10',
    '/usr/lib/python3.10/lib-dynload',
    '/usr/local/lib/python3.10/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/root/.local' (exists)
USER_SITE: '/root/.local/lib/python3.10/site-packages' (doesn't exist)
ENABLE_USER_SITE: True
root@a326e8cb61dd:~# poetry config virtualenvs.in-project true
root@a326e8cb61dd:~# poetry new first
Created package first in first
root@a326e8cb61dd:~# cd first
root@a326e8cb61dd:~/first# poetry install
Creating virtualenv first in /root/first/.venv
Updating dependencies
Resolving dependencies... (6.0s)

Writing lock file

Package operations: 8 installs, 0 updates, 0 removals

  • Installing pyparsing (3.0.8)
  • Installing attrs (21.4.0)
  • Installing more-itertools (8.12.0)
  • Installing packaging (21.3)
  • Installing pluggy (0.13.1)
  • Installing py (1.11.0)
  • Installing wcwidth (0.2.5)
  • Installing pytest (5.4.3)

Installing the current project: first (0.1.0)

mr-xeroth · 2022-04-29T04:42:46Z

@abn So I dug through the env.py. The difference is obvious. Any advice on how to update my system python's config with proper path values?

docker:

$ poetry new foo
Created package foo in foo
$ cd foo
$ poetry install

4cwd: /home/useless/foo

2cwd: /home/useless/foo

1venv_path: /home/useless/foo/.venv
Creating virtualenv foo in /home/useless/foo/.venv

get_paths() from script
('{"stdlib": "/usr/lib/python3.10", "platstdlib": '
 '"/home/useless/foo/.venv/lib/python3.10", "purelib": '
 '"/home/useless/foo/.venv/lib/python3.10/site-packages", "platlib": '
 '"/home/useless/foo/.venv/lib/python3.10/site-packages", "include": '
 '"/usr/include/python3.10", "platinclude": "/usr/include/python3.10", '
 '"scripts": "/home/useless/foo/.venv/bin", "data": '
 '"/home/useless/foo/.venv"}\n')

self._paths:
{'data': '/home/useless/foo/.venv',
 'include': '/usr/include/python3.10',
 'platinclude': '/usr/include/python3.10',
 'platlib': '/home/useless/foo/.venv/lib/python3.10/site-packages',
 'platstdlib': '/home/useless/foo/.venv/lib/python3.10',
 'purelib': '/home/useless/foo/.venv/lib/python3.10/site-packages',
 'scripts': '/home/useless/foo/.venv/bin',
 'stdlib': '/usr/lib/python3.10'}

self._purelib: /home/useless/foo/.venv/lib/python3.10/site-packages
Updating dependencies
Resolving dependencies... (2.4s)

Writing lock file

Package operations: 8 installs, 0 updates, 0 removals

  • Installing pyparsing (3.0.8)
  • Installing attrs (21.4.0)
  • Installing more-itertools (8.12.0)
  • Installing packaging (21.3)
  • Installing pluggy (0.13.1)
  • Installing py (1.11.0)
  • Installing wcwidth (0.2.5)
  • Installing pytest (5.4.3)

Installing the current project: foo (0.1.0)
self.usersite None

self.purelib: /home/useless/foo/.venv/lib/python3.10/site-packages

SitePackages.path: /home/useless/foo/.venv/lib/python3.10/site-packages

SidePackage.candidates: [PosixPath('/home/useless/foo/.venv/lib/python3.10/site-packages')]

env path OK "/home/useless/foo/.venv/lib/python3.10/site-packages/foo-0.1.0.dist-info"
Installing the current project: foo (0.1.0)

my system:

user@comp:~$ poetry new foo
Created package foo in foo
user@comp:~$ cd foo
user@comp:~/foo$ poetry install

4cwd: /home/user/foo

2cwd: /home/user/foo

1venv_path: /home/user/foo/.venv
Creating virtualenv foo in /home/user/foo/.venv

get_paths() from script
('{"stdlib": "/usr/lib/python3.10", "platstdlib": "/usr/lib/python3.10", '
 '"purelib": "/usr/local/lib/python3.10/dist-packages", "platlib": '
 '"/usr/local/lib/python3.10/dist-packages", "include": '
 '"/usr/include/python3.10", "platinclude": "/usr/include/python3.10", '
 '"scripts": "/usr/local/bin", "data": "/usr/local"}\n')

self._paths:
{'data': '/usr/local',
 'include': '/usr/include/python3.10',
 'platinclude': '/usr/include/python3.10',
 'platlib': '/usr/local/lib/python3.10/dist-packages',
 'platstdlib': '/usr/lib/python3.10',
 'purelib': '/usr/local/lib/python3.10/dist-packages',
 'scripts': '/usr/local/bin',
 'stdlib': '/usr/lib/python3.10'}

self._purelib: /usr/local/lib/python3.10/dist-packages
Updating dependencies
Resolving dependencies... (2.4s)

Writing lock file

Package operations: 0 installs, 8 updates, 0 removals

  • Updating pyparsing (2.4.7 /usr/lib/python3/dist-packages -> 3.0.8)
  • Updating attrs (21.4.0 /home/user/.local/lib/python3.10/site-packages -> 21.4.0)
  • Updating more-itertools (8.10.0 /usr/lib/python3/dist-packages -> 8.12.0)
  • Updating packaging (21.3 /home/user/.local/lib/python3.10/site-packages -> 21.3)
  • Updating pluggy (0.13.1 /home/user/.local/lib/python3.10/site-packages -> 0.13.1)
  • Updating py (1.11.0 /home/user/.local/lib/python3.10/site-packages -> 1.11.0)
  • Updating wcwidth (0.2.5 /home/user/.local/lib/python3.10/site-packages -> 0.2.5)
  • Updating pytest (5.4.3 /home/user/.local/lib/python3.10/site-packages -> 5.4.3)

Installing the current project: foo (0.1.0)
self.usersite None

self.purelib: /usr/local/lib/python3.10/dist-packages

SitePackages.path: /usr/local/lib/python3.10/dist-packages

SidePackage.candidates: [PosixPath('/usr/local/lib/python3.10/dist-packages')]

env path ERR "/usr/local/lib/python3.10/dist-packages/foo.pth"
  - Failed to create foo.pth for /home/user/foo
  - Failed to find a suitable script installation directory for /home/user/foo

env path OK "/usr/local/lib/python3.10/dist-packages/foo-0.1.0.dist-info"

env path ERR "/usr/local/lib/python3.10/dist-packages/foo-0.1.0.dist-info"

  OSError

  Unable to access any of "/usr/local/lib/python3.10/dist-packages/foo-0.1.0.dist-info"

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:273 in _path_method_wrapper
       269│             print('\nenv path OK', paths_csv(candidates))
       270│             return results
       271│ 
       272│         print('\nenv path ERR', paths_csv(candidates))
    →  273│         raise OSError("Unable to access any of {}".format(paths_csv(candidates)))
       274│ 
       275│     def write_text(self, path, *args, **kwargs):  # type: (Path, *Any, **Any) -> Path
       276│         return self._path_method_wrapper(path, "write_text", *args, **kwargs)[0]
       277│

refi64 · 2022-04-29T14:47:04Z

I encountered the same issue, and I think some mix of the following fixed it:

Make sure the virtualenv package inside your Poetry venv is at least version 20.10.0, so it has this commit
rm -rf ~/.local/share/virtualenv/py_info/ to clear the cached path information

I believe this is a distro packaging bug (Fedora bug report). In my case, Fedora 36, changed the way it tried to install packages to /usr/local instead of /usr, but that broke venv creation. Although I have the fixed packages, I'm guessing that the cached path information still had the old values.

The related Python bug mentions Ubuntu, so they might have encountered a similar problem. If so, it worked in Docker because there was no virtualenv py_info cache there.

mr-xeroth · 2022-04-29T15:54:18Z

@refi64 I've installed virtualenv-20.14.1, removed /py_info to the same result. Thanks for the input.

abn · 2022-04-29T16:17:19Z

@mr-xeroth in your case it looks like Poetry executes your system python instead of the venv python. As for @refi64 's comment you might want to make sure you are upgrading virtualenv in your Poetry environment and not just your systen environment.

Something like this:

$ cat $(which poetry) | head -n 1
#!/home/abn/.local/pipx/venvs/poetry/bin/python
$ /home/abn/.local/pipx/venvs/poetry/bin/python -m pip install --upgrade virtualenv

kavdev · 2022-06-07T18:39:37Z

@refi64 I've been racking my brain all morning on this... clearing the cache saved me. Thanks for that!

mohitt · 2022-06-10T03:49:18Z

Facing the same issue, if I create a virtualenv manually on ubuntu 22.04. output of my python -m site is

sys.path = [
    '/home/mthakral/ti/vryno/vocrm',
    '/usr/local/lib/python310.zip',
    '/usr/local/lib/python3.10',
    '/usr/local/lib/python3.10/lib-dynload',
    '/home/mthakral/.virtualenvs/vocrm-gvop/lib/python3.10/site-packages',
    '/home/mthakral/ti/vryno/vocore',
]
USER_BASE: '/home/mthakral/.local' (exists)
USER_SITE: '/home/mthakral/.local/lib/python3.10/site-packages' (exists)
ENABLE_USER_SITE: False

where as if I create a new poetry project and try doing poetry install. I get following exception.

The virtual environment found in /home/mthakral/.cache/pypoetry/virtualenvs/voreleaser-kWmOh-cm-py3.10 seems to be broken.
Recreating virtualenv voreleaser-kWmOh-cm-py3.10 in /home/mthakral/.cache/pypoetry/virtualenvs/voreleaser-kWmOh-cm-py3.10
Installing dependencies from lock file

Package operations: 0 installs, 8 updates, 1 removal

  • Removing colorama (0.4.4): Failed

  EnvCommandError

  Command ['pip', 'uninstall', 'colorama', '-y'] errored with the following return code 2, and output:
  Found existing installation: colorama 0.4.4
  Uninstalling colorama-0.4.4:
  ERROR: Exception:
  Traceback (most recent call last):
    File "/usr/lib/python3.10/shutil.py", line 813, in move
      os.rename(src, real_dst)
  PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/colorama' -> '/tmp/pip-uninstall-tg1yf371'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/mthakral/.local/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
      status = run_func(*args)
    File "/home/mthakral/.local/lib/python3.10/site-packages/pip/_internal/commands/uninstall.py", line 97, in run
      uninstall_pathset = req.uninstall(
    File "/home/mthakral/.local/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 638, in uninstall
      uninstalled_pathset.remove(auto_confirm, verbose)
    File "/home/mthakral/.local/lib/python3.10/site-packages/pip/_internal/req/req_uninstall.py", line 369, in remove
      moved.stash(path)
    File "/home/mthakral/.local/lib/python3.10/site-packages/pip/_internal/req/req_uninstall.py", line 267, in stash
      renames(path, new_path)
    File "/home/mthakral/.local/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 305, in renames
      shutil.move(old, new)
    File "/usr/lib/python3.10/shutil.py", line 831, in move
      rmtree(src)
    File "/usr/lib/python3.10/shutil.py", line 722, in rmtree
      _rmtree_safe_fd(fd, path, onerror)
    File "/usr/lib/python3.10/shutil.py", line 678, in _rmtree_safe_fd
      onerror(os.unlink, fullname, sys.exc_info())
    File "/usr/lib/python3.10/shutil.py", line 676, in _rmtree_safe_fd
      os.unlink(entry.name, dir_fd=topfd)
  PermissionError: [Errno 13] Permission denied: 'ansitowin32.py'


  at ~/.poetry/lib/poetry/utils/env.py:1195 in _run
      1191│                 output = subprocess.check_output(
      1192│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1193│                 )
      1194│         except CalledProcessError as e:
    → 1195│             raise EnvCommandError(e, input=input_)
      1196│
      1197│         return decode(output)
      1198│
      1199│     def execute(self, bin, *args, **kwargs):
     
     ```
     
     if i check my `python -m site` I see following 
     
     ```
     ❯ python -m site
sys.path = [
    '/home/mthakral/ti/vryno/voreleaser',
    '/usr/lib/python310.zip',
    '/usr/lib/python3.10',
    '/usr/lib/python3.10/lib-dynload',
    '/home/mthakral/.local/lib/python3.10/site-packages',
    '/usr/local/lib/python3.10/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/mthakral/.local' (exists)
USER_SITE: '/home/mthakral/.local/lib/python3.10/site-packages' (exists)
ENABLE_USER_SITE: True

output of my poetry debug info


Poetry
Version: 1.1.13
Python:  3.10.4

Virtualenv
Python:         3.10.4
Implementation: CPython
Path:           /home/mthakral/.cache/pypoetry/virtualenvs/voreleaser-kWmOh-cm-py3.10
Valid:          False

System
Platform: linux
OS:       posix
Python:   /usr

Basically sys.path as dist-packages . I am not sure why.

cemsbr · 2022-06-14T20:33:00Z

I was having a lot of problems and some error messages like the ones reported here. I saw pypa/setuptools#3278 and the workaround was setting the env var SETUPTOOLS_USE_DISTUTILS=stdlib.

akhildevelops · 2022-08-30T16:59:53Z

I was having a lot of problems and some error messages like the ones reported here. I saw pypa/setuptools#3278 and the workaround was setting the env var SETUPTOOLS_USE_DISTUTILS=stdlib.

This helped me. Thanks

neersighted · 2022-08-30T23:15:57Z

Closing this for now as this is related to distro/setuptools bugs that are either resolved or documented (e.g. on the installer repo).

jedie · 2022-11-13T20:16:39Z

I ran into the same problem. Work-a-round is to create the venv first, e.g.:

python3 -m venv .venv
poetry install

Then poetry used the existing .venv and the bug pypa/setuptools#3278 didn't happen.

neersighted · 2022-11-13T20:52:11Z

This is documented/well known as #6371; venv works as it is patched by the distro, but virtualenv does not as the distro's patches result in a incoherent sysconfig.

See: * python-poetry/poetry#5490 (comment) * python-poetry/poetry#6371

github-actions · 2024-03-01T00:18:37Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

mr-xeroth added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Apr 25, 2022

neersighted closed this as completed Aug 30, 2022

mkniewallner removed the status/triage This issue needs to be triaged label Sep 18, 2022

jedie mentioned this issue Nov 13, 2022

Permission denied when attempting poetry self update --preview #4426

Closed

3 tasks

jedie pushed a commit to boxine/bx_django_utils that referenced this issue Nov 21, 2022

Add work-a-round for "poetry install" bug

d649dba

See: * python-poetry/poetry#5490 (comment) * python-poetry/poetry#6371

jedie added a commit to jedie/cookiecutter_templates that referenced this issue Nov 21, 2022

Add work-a-round for "poetry install" bug

d9a355d

See: * python-poetry/poetry#5490 (comment) * python-poetry/poetry#6371

jedie mentioned this issue Nov 21, 2022

Add work-a-round for "poetry install" bug jedie/cookiecutter_templates#10

Merged

jedie added a commit to jedie/django_example that referenced this issue Nov 21, 2022

Add work-a-round for "poetry install" bug

32f4265

See: * python-poetry/poetry#5490 (comment) * python-poetry/poetry#6371

jedie mentioned this issue Nov 21, 2022

Add work-a-round for "poetry install" bug jedie/django_example#3

Merged

sync-by-unito bot mentioned this issue Aug 28, 2023

Bump Project Python version to 3.10 toddbirchard/ghost-webhook-api#296

Closed

github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

the simplest 'poetry install' fails #5490

the simplest 'poetry install' fails #5490

mr-xeroth commented Apr 25, 2022 •

edited

abn commented Apr 25, 2022

mr-xeroth commented Apr 25, 2022

mr-xeroth commented Apr 29, 2022

refi64 commented Apr 29, 2022

mr-xeroth commented Apr 29, 2022

abn commented Apr 29, 2022

kavdev commented Jun 7, 2022

mohitt commented Jun 10, 2022

cemsbr commented Jun 14, 2022

akhildevelops commented Aug 30, 2022

neersighted commented Aug 30, 2022

jedie commented Nov 13, 2022

neersighted commented Nov 13, 2022

github-actions bot commented Mar 1, 2024

the simplest 'poetry install' fails #5490

the simplest 'poetry install' fails #5490

Comments

mr-xeroth commented Apr 25, 2022 • edited

abn commented Apr 25, 2022

mr-xeroth commented Apr 25, 2022

mr-xeroth commented Apr 29, 2022

refi64 commented Apr 29, 2022

mr-xeroth commented Apr 29, 2022

abn commented Apr 29, 2022

kavdev commented Jun 7, 2022

mohitt commented Jun 10, 2022

cemsbr commented Jun 14, 2022

akhildevelops commented Aug 30, 2022

neersighted commented Aug 30, 2022

jedie commented Nov 13, 2022

neersighted commented Nov 13, 2022

github-actions bot commented Mar 1, 2024

mr-xeroth commented Apr 25, 2022 •

edited