Files for a src-style project installed in editable or develop mode

[jaraco]

In GitLab by @sinoroc on Feb 23, 2020, 18:34

In the case of a project with a src-style directory structure that is installed in develop or editable mode, it is not immediately possible to access the files.

.
├── setup.py
└── src
    └── mypackage
        └── __init__.py

#!/usr/bin/env python3

import setuptools

setuptools.setup(
    name='MyProject',
    version='0.0.0.dev0',
    packages=setuptools.find_packages(where='src'),
    package_dir={
        '': 'src',
    },
)

import importlib_metadata

print("importlib_metadata.__version__", importlib_metadata.__version__)

dist = importlib_metadata.distribution('MyProject')
print("dist.locate_file('')", dist.locate_file(''))
for file_ in dist.files:
    print("file_", file_)
for file_ in dist.files:
    print("file_", file_)
    print(file_.read_text())

$ ./setup.py develop
running develop
running egg_info
creating src/MyProject.egg-info
writing src/MyProject.egg-info/PKG-INFO
writing dependency_links to src/MyProject.egg-info/dependency_links.txt
writing top-level names to src/MyProject.egg-info/top_level.txt
writing manifest file 'src/MyProject.egg-info/SOURCES.txt'
reading manifest file 'src/MyProject.egg-info/SOURCES.txt'
writing manifest file 'src/MyProject.egg-info/SOURCES.txt'
running build_ext
Creating /tmp/tmp.BzMk6xAF0S/.venv/lib/python3.6/site-packages/MyProject.egg-link (link to src)
MyProject 0.0.0.dev0 is already the active version in easy-install.pth

Installed /tmp/tmp.BzMk6xAF0S/src
Processing dependencies for MyProject==0.0.0.dev0
Finished processing dependencies for MyProject==0.0.0.dev0

$ python -c 'import mypackage'
importlib_metadata.__version__ 1.5.0
dist.locate_file('') /tmp/tmp.BzMk6xAF0S/src
file_ setup.py
file_ src/MyProject.egg-info/PKG-INFO
file_ src/MyProject.egg-info/SOURCES.txt
file_ src/MyProject.egg-info/dependency_links.txt
file_ src/MyProject.egg-info/top_level.txt
file_ src/mypackage/__init__.py
file_ setup.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmp.BzMk6xAF0S/src/mypackage/__init__.py", line 11, in <module>
    print(file_.read_text())
  File "/tmp/tmp.BzMk6xAF0S/.venv/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 140, in read_text
    with self.locate().open(encoding=encoding) as stream:
  File "/usr/lib/python3.6/pathlib.py", line 1183, in open
    opener=self._opener)
  File "/usr/lib/python3.6/pathlib.py", line 1037, in _opener
    return self._accessor.open(self, flags, mode)
  File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp.BzMk6xAF0S/src/setup.py'

This is obviously a special case, the combination of editable (or develop) mode with setuptools package_dir seems quite tricky to deal with.

[jaraco]

In GitLab by @blueyed on Mar 17, 2020, 12:34

mentioned in commit blueyed/importlib_metadata@5d73789b0619d5ac53dfb2966902c13b24d31e2b

[jaraco]

In GitLab by @blueyed on Mar 17, 2020, 12:34

mentioned in merge request !114

[jaraco]

In GitLab by @blueyed on Mar 17, 2020, 12:35

This is also a problem when not using "wheel".
I've created https://gitlab.com/python-devs/importlib_metadata/-/merge_requests/114, which addresses this, but is only a "quick hack" / not finished, and would appreciate feedback over there.

[jaraco]

Although some work was done on related issues, I believe this one remains. I welcome efforts to revive the effort or come up with a new approach.

[abravalheri]

This probably works (somehow) for pip install -e . (when PEP 660 is used), right?
importlib.metadata should be able to find the .dist-info directory because it is added to sys.path

(Everything that depends on the RECORD file will be weird, but that is a general problem with editable installs)