BUG: 8.7.0 removes code-block:: ipython pygment lexer support

[larsoner]

Installed via pip / PyPI -- previously this RST worked:

.. code-block:: ipython

    In [1]: %matplotlib qt

but now we get:

/home/circleci/project/doc/install/advanced.rst:33: WARNING: Pygments lexer name 'ipython' is not known

At least this is what I think is happening in our CircleCI build today (which fails because we treat warnings as errors), we haven't changed the RST lines in question at all recently:

https://app.circleci.com/pipelines/github/mne-tools/mne-python/17137/workflows/27c6f253-b7a7-4b67-9c2e-db9aa4e925fc/jobs/50745?invite=true#step-112-532

And 8.7.0 is being installed properly in the env:

https://app.circleci.com/pipelines/github/mne-tools/mne-python/17137/workflows/27c6f253-b7a7-4b67-9c2e-db9aa4e925fc/jobs/50745?invite=true#step-112-532

Changing this to .. code-block:: ipythonconsole does not seem to help.

Is this an intentional change? Or is this lexer meant to be installed a different way? I didn't see anything in these pages at least:

• https://ipython.readthedocs.io/en/stable/whatsnew/version8.html#ipython-8-7-0
• https://ipython.readthedocs.io/en/stable/development/lexer.html

[jenshnielsen]

I ran into the same problem. I think this is due to the Pygments entry points not being correctly installed with 8.6.0

Compare

Python 3.10.8 | packaged by conda-forge | (main, Nov  4 2022, 13:42:51) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.6.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from importlib.metadata import entry_points

In [2]: entry_points(group="pygments.lexers")
Out[2]:
[EntryPoint(name='ipython', value='IPython.lib.lexers:IPythonLexer', group='pygments.lexers'),
 EntryPoint(name='ipython3', value='IPython.lib.lexers:IPython3Lexer', group='pygments.lexers'),
 EntryPoint(name='ipythonconsole', value='IPython.lib.lexers:IPythonConsoleLexer', group='pygments.lexers')]

to this

Python 3.10.8 | packaged by conda-forge | (main, Nov  4 2022, 13:42:51) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.7.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from importlib.metadata import entry_points

In [2]: entry_points(group="pygments.lexers")
Out[2]: []

I suspect this is related to #13842 as setuptools probably doesn't merge the entry points configured in setup.py with the ones configured in setup.cfg

[matthewfeickert]

Agree that this seems to be a bug that snuck into v8.7.0 (it happens!), but we can easily get around this for the time being by just having

ipython!=8.7.0

in our setup.{py,cfg} files and build requirements.txts to disallow this version. As this will get fixed by the next release (I would assume) then you're basically done as you don't have to uncap later. 👍

[frenzymadness]

I also have the same problem because the not available lexer breaks the build of TestSlide package. Unfortunately, it seems that I won't be able to update ipython in Fedora Linux to 8.7.0 because of this issue.

[frenzymadness]

I want to prepare a fix for this issue. What should be the single point of truth for the entrypoints? Setup.cfg or setup.py?

[jenshnielsen]

@frenzymadness I think it needs to be setup.py #13842 moved some of the entry points to setup.py since they are dynamically generated and that is apparently only supported in setup.py

[bollwyvl]

ah, was behind on this discussion, this would have been a great first contribution! entry_points are dark magic indeed, and packaging bugs are never fun in general, but the fix is not that large:

#13848

hopefully that patch will be sufficient for any that would like to ship 8.7.0 with the pygments integration, if your build system allows patches.

to that end: i've applied something similar on conda-forge/ipython-feedstock#182, but also had to deal with wanting to do minimal builds, so we've also added the python-minor entry_point, e.g. ipython3.8, as a separate package, _ipython_minor_entry_point-feedstock

[matthew-brett]

Thanks for the quick fix. Any chance of a point release with the fix?

[choldgraf]

♥️ for fixing this