test_entry_point failing in some Arch environments

[FFY00]

When packaging for Arch Linux, I am getting the following error. I would guess it is due to the entrypoint API changes in importlib_metadata, but I don't have time to explore it right now, I'll try to later. If this is a known issue, please let me know.

============================= test session starts ==============================
platform linux -- Python 3.9.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /build/python-keyring/src/keyring, configfile: pytest.ini
plugins: mypy-0.8.1, black-0.3.12, flake8-1.0.7, cov-2.12.1
collected 77 items

keyring/core.py .                                                        [  1%]
keyring/backends/chainer.py .                                            [  2%]
keyring/backends/fail.py .                                               [  3%]
keyring/backends/null.py .                                               [  5%]
keyring/util/__init__.py .                                               [  6%]
keyring/util/properties.py ..                                            [  9%]
tests/test_core.py .                                                     [ 10%]
tests/test_errors.py .                                                   [ 11%]
tests/test_integration.py ...                                            [ 15%]
tests/test_multiprocess.py XX                                            [ 18%]
tests/test_packaging.py F.                                               [ 20%]
tests/backends/test_SecretService.py sssssssssss.                        [ 36%]
tests/backends/test_Windows.py ssssssssssssss                            [ 54%]
tests/backends/test_chainer.py ..                                        [ 57%]
tests/backends/test_kwallet.py ssssssssssssssssssssss                    [ 85%]
tests/backends/test_macOS.py sssssssssss                                 [100%]

=================================== FAILURES ===================================
_______________________________ test_entry_point _______________________________

    def test_entry_point():
        """
        Keyring provides exactly one 'keyring' console script
        that's a callable.
        """
        matches = metadata.entry_points(group='console_scripts', name='keyring')
>       (script,) = matches
E       ValueError: too many values to unpack (expected 1)

tests/test_packaging.py:12: ValueError
=========================== short test summary info ============================
FAILED tests/test_packaging.py::test_entry_point - ValueError: too many value...
============= 1 failed, 16 passed, 58 skipped, 2 xpassed in 0.44s ==============

[jaraco]

It's not a known issue. That test is apparently revealing that there is more than one package presenting the keyring console_script. I'd use the metadata package to inspect why that is in that environment (what is [ep.dist for ep in matches]?).

[musicinmybrain]

I ran into this while trying to update the Fedora Linux python-keyring from 21.8.0 to 23.5.0. I’m not sure quite what the problem is. I see two matches:

• [ep.value for ep in …] is ['keyring.cli:main', 'keyring.cli:main']
• [ep.load() for ep in …] is [<importlib_metadata.PathDistribution object at 0x7f57c941f490>, <importlib_metadata.PathDistribution object at 0x7f57c941f340>]; that is, they are associated with the same distribution
• [ep.load() for ep in …] is [<function main at 0x7f8506479bd0>, <function main at 0x7f8506479bd0>]; that is, they resolve to the same function

This seems to be somehow an artifact of the RPM build environment, in which the package is installed into a “buildroot” directory (and PYTHONPATH is set for the tests) rather than into the system site-packages.

When I skip the test and install the resulting RPM, then look at importlib_metadata.entry_points(group='console_scripts', name='keyring') in a Python REPL, only one EntryPoint is found. I suspect the situation in Arch may be similar.

For now I’ll just keep skipping the test, but I’m happy to test out any ideas.

[jaraco]

• [ep.load() for ep in …] is [<importlib_metadata.PathDistribution object at 0x7f57c941f490>, <importlib_metadata.PathDistribution object at 0x7f57c941f340>]; that is, they are associated with the same distribution

Those look like two different distributions to me (one located at f490 and the other at f340).

Once you have a PathDistribution, you can inspect it's _path property, which will tell you from where the distribution is being derived.

Also, if you have importlib_metadata 3.5.0 or importlib.metadata on Python 3.10, the entry points will always only return entry points for unique distributions, so I expect this problem to go away on its own as those older versions are retired.

Oh, that's interesting. I notice that keyring does not rely on importilib.metadata, but instead relies on importlib_metadata >= 3.6, so I'd expect keyring not to have the reported issue.

What do you get for importlib_metadata.version('importlib_metadata')?

[musicinmybrain]

• [ep.load() for ep in …] is [<importlib_metadata.PathDistribution object at 0x7f57c941f490>, <importlib_metadata.PathDistribution object at 0x7f57c941f340>]; that is, they are associated with the same distribution

Those look like two different distributions to me (one located at f490 and the other at f340).

You’re right. That was a reading failure on my part. Plus, that should have been [ep.dist for ep in …].

Once you have a PathDistribution, you can inspect it's _path property, which will tell you from where the distribution is being derived.

Thanks. This is helpful. For [ep.dist._path for ep in …], I get:

[PosixPath('/builddir/build/BUILD/keyring-23.5.0/keyring.egg-info'), PosixPath('/builddir/build/BUILDROOT/python-keyring-23.5.0-1.fc37.x86_64/usr/lib/python3.10/site-packages/keyring-23.5.0-py3.10.egg-info')]

which makes sense. One is the “built” but not yet installed package, and the other is the copy installed in the buildroot (what will actually be packaged).

Also, if you have importlib_metadata 3.5.0 or importlib.metadata on Python 3.10, the entry points will always only return entry points for unique distributions, so I expect this problem to go away on its own as those older versions are retired.

Oh, that's interesting. I notice that keyring does not rely on importilib.metadata, but instead relies on importlib_metadata >= 3.6, so I'd expect keyring not to have the reported issue.

What do you get for importlib_metadata.version('importlib_metadata')?

I’m building with Python 3.10.4 and importlib-metadata 4.11.3.

---

Looking at [ep.dist._path for ep in …] pointed me to an easy fix or workaround, which is to do

rm -rvf *.egg-info

before running the tests. I don’t know Arch packaging, but I suspect something similar will work there.

[musicinmybrain]

I also suspect, but haven’t yet confirmed, that this would go away without needing a workaround if the Fedora package were switched to the newer pyproject-rpm-macros approach.

[musicinmybrain]

I also suspect, but haven’t yet confirmed, that this would go away without needing a workaround if the Fedora package were switched to the newer pyproject-rpm-macros approach.

I do still need the workaround with pyproject-rpm-macros, although it becomes:

rm -rvf *.egg-info *.dist-info

because the build process now produces keyring.dist-info, and the PyPI tarball already contains keyring.egg-info.

[Apteryks]

I also get this issue on GNU Guix:

tests/test_packaging.py F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

    def test_entry_point():
        """
        Keyring provides exactly one 'keyring' console script
        that's a callable.
        """
        matches = metadata.entry_points(group='console_scripts', name='keyring')
>       (script,) = matches
E       ValueError: too many values to unpack (expected 1)

tests/test_packaging.py:11: ValueError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[12] > /tmp/guix-build-python-keyring-23.9.3.drv-0/keyring-23.9.3/tests/test_packaging.py(11)test_entry_point()
-> (script,) = matches
   6 frames hidden (try 'help hidden_frames')
> /tmp/guix-build-python-keyring-23.9.3.drv-0/keyring-23.9.3/tests/test_packaging.py(11)

   5     def test_entry_point():                                                                          
   6         """                                                                                          
   7         Keyring provides exactly one 'keyring' console script                                        
   8         that's a callable.                                                                           
   9         """                                                                                          
  10         matches = metadata.entry_points(group='console_scripts', name='keyring')                     
  11  ->     (script,) = matches                                                                          
  12         assert callable(script.load())                                                               
(Pdb++) matches
[EntryPoint(name='keyring', value='keyring.cli:main', group='console_scripts'), EntryPoint(name='keyring', value='keyring.cli:main', group='console_scripts')]

The direct dependencies are at: `python-importlib-metadata@4.11.3 python-pytest@6.2.5 python-secretstorage@3.3.1

• python-setuptools-scm@6.3.2 python-setuptools@64.0.3 python-toml@0.10.2`, using Python 3.9.9.

The workaround posted in the previous message, rm -rvf *.egg-info *.dist-info, works for me too.

[jaraco]

I was able to replicate this issue thus:

 keyring main $ py -m venv .venv
 keyring main $ py -m pip install -q '.[testing]'
 keyring main $ py -m pytest -p no:cov -k test_entry_point
================================================================= test session starts ==================================================================
platform darwin -- Python 3.11.0, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/jaraco/code/jaraco/keyring, configfile: pytest.ini
plugins: black-0.3.12, checkdocs-2.9.0, flake8-1.1.1, mypy-0.10.0, enabler-1.3.0
collected 229 items / 227 deselected / 2 selected                                                                                                      

tests/test_packaging.py F                                                                                                                        [100%]

======================================================================= FAILURES =======================================================================
___________________________________________________________________ test_entry_point ___________________________________________________________________

    def test_entry_point():
        """
        Keyring provides exactly one 'keyring' console script
        that's a callable.
        """
        matches = metadata.entry_points(group='console_scripts', name='keyring')
>       (script,) = matches
E       ValueError: too many values to unpack (expected 1)

tests/test_packaging.py:11: ValueError
=============================================================== short test summary info ================================================================
FAILED tests/test_packaging.py::test_entry_point - ValueError: too many values to unpack (expected 1)
========================================================== 1 failed, 227 deselected in 0.18s ===========================================================

The reason the error doesn't occur when running under tox is because tox installs the package as editable and thus there's only one metadata to be found.

Next I'll try to find out why importlib_metadata isn't providing unique values.

[jaraco]

The issue is that the _normalized_name coming out of the egg still has egg in it:

 keyring main $ py -m pytest -p no:cov -k test_entry_point --pdb
================================================================= test session starts ==================================================================
platform darwin -- Python 3.11.0, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/jaraco/code/jaraco/keyring, configfile: pytest.ini
plugins: black-0.3.12, checkdocs-2.9.0, flake8-1.1.1, mypy-0.10.0, enabler-1.3.0
collected 229 items / 227 deselected / 2 selected                                                                                                      

tests/test_packaging.py F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

    def test_entry_point():
        """
        Keyring provides exactly one 'keyring' console script
        that's a callable.
        """
        matches = metadata.entry_points(group='console_scripts', name='keyring')
>       (script,) = matches
E       ValueError: too many values to unpack (expected 1)

tests/test_packaging.py:11: ValueError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/jaraco/code/jaraco/keyring/tests/test_packaging.py(11)test_entry_point()
-> (script,) = matches
(Pdb) eps = list(matches)
(Pdb) eps
[EntryPoint(name='keyring', value='keyring.cli:main', group='console_scripts'), EntryPoint(name='keyring', value='keyring.cli:main', group='console_scripts')]
(Pdb) eps[0].dist._normalized_name
'keyring.egg'
(Pdb) eps[1].dist._normalized_name
'keyring'

[musicinmybrain]

Thanks for looking into this! The fix works as advertised for me; I no longer need a workaround on Fedora versions where importlib-metadata is new enough to satisfy the updated requirement in v23.10.0.