Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for docutils 0.19 #8

Closed
Qeole opened this issue Aug 2, 2022 · 1 comment · Fixed by #9
Closed

Update for docutils 0.19 #8

Qeole opened this issue Aug 2, 2022 · 1 comment · Fixed by #9

Comments

@Qeole
Copy link

Qeole commented Aug 2, 2022

Description

Description mostly copied from m2r2's CrossNox/m2r2#49:

ErrorString and SafeString are deprecated and have been removed in docutils 0.19. We could replace ErrorString with io.error_string and SafeString with Python's builtin str() (although this seems breaking for Python 2.x).

See docutils:HISTORY.txt and docutils:utils/error_reporting.py for more information.

Details

Trying to use sphinx-mdinclude with docutils 0.19, I got:

ImportError while loading conftest '/tmp/openapi/tests/conftest.py'.                                                                                                                                     
tests/conftest.py:9: in <module>                                                                                                                                                                         
    from sphinxcontrib.openapi import utils                                                                                                                                                              
.tox/py/lib/python3.10/site-packages/sphinxcontrib/openapi/__init__.py:13: in <module>                                                                                                                   
    from sphinxcontrib.openapi import renderers, directive                                                                                                                                               
.tox/py/lib/python3.10/site-packages/sphinxcontrib/openapi/renderers/__init__.py:5: in <module>                                                                                                          
    from ._httpdomain import HttpdomainRenderer                                                                                                                                                          
.tox/py/lib/python3.10/site-packages/sphinxcontrib/openapi/renderers/_httpdomain.py:11: in <module>                                                                                                      
    import sphinx_mdinclude                                                                                                                                                                              
.tox/py/lib/python3.10/site-packages/sphinx_mdinclude/__init__.py:12: in <module>                                                                                                                        
    from .sphinx import setup                                                                                                                                                                            
.tox/py/lib/python3.10/site-packages/sphinx_mdinclude/sphinx.py:9: in <module>                                                                                                                           
    from docutils.core import ErrorString                                                                                                                                                                
E   ImportError: cannot import name 'ErrorString' from 'docutils.core' (/tmp/openapi/.tox/py/lib/python3.10/site-packages/docutils/core.py)                                                              
ERROR: InvocationError for command /tmp/openapi/.tox/py/bin/python -m pytest tests/ --strict (exited with code 4)
  • OS: Ubuntu 22.04
  • Python version: 3.10.4
  • Project version: sphinx_mdinclude==0.5.1
  • Can you repro on main?
    Haven't tried, but the import is still here
  • Can you repro in a clean virtualenv?
    Ran with tox, I'm not super familiar with Python's virtual environments but can help reproduce if it were necessary.
@flokli
Copy link

flokli commented Aug 25, 2022

I used this following patch to get rid of the import errors:

diff --git a/sphinx_mdinclude/sphinx.py b/sphinx_mdinclude/sphinx.py
index 44e0f87..d006feb 100644
--- a/sphinx_mdinclude/sphinx.py
+++ b/sphinx_mdinclude/sphinx.py
@@ -6,10 +6,9 @@ import os
 import os.path
 
 from docutils import io, nodes, statemachine, utils
-from docutils.core import ErrorString
+from docutils.utils.error_reporting import ErrorString, SafeString
 from docutils.parsers import rst
 from docutils.parsers.rst import directives as rst_directives
-from docutils.utils import SafeString
 
 from . import RestMarkdown
 

Note this will probably only fix docutils 0.19, but might break with older versions - so if sphinx-mdinclude intends to keep compat with both docutils versions for the time being, there would probably need to be some conditional code.

amyreese added a commit that referenced this issue Aug 26, 2022
This attempts to import ErrorString and SafeString from their new
locations in docutils, and falls back to their previous locations if the
import fails.

Fixes #8
amyreese added a commit that referenced this issue Aug 26, 2022
This attempts to import ErrorString and SafeString from their new
locations in docutils, and falls back to their previous locations if the
import fails.

Fixes #8
qmonnet added a commit to cilium/cilium that referenced this issue Jan 17, 2023
Once upon a time, Cilium docs used the openapi Sphinx add-on to generate
its API reference based on the code. And things were good.

One day, Dependabot raised a security alert, stating that Mistune v2.0.2
was vulnerable to catastrophic backtracking [0] - this is a regex
parsing thing. Mistune was a dependency to m2r, an add-on to parse
Markdown in Sphinx, which in turn was a dependency to openapi.

The easy path would have been to update m2r to use the latest, fixed
Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and
also it was no longer in development.

There was a fork, m2r2, which had little activity, and would avoid the
security issue by very simply pinning the Mistune version to 0.8.4
(which would either fail to build Cilium's reference correctly, or bring
some incompatibilities with other dependencies, at this point the
narrator does not remember for sure).

There was a fork of the fork, sphinx-mdinclude. We could use that
project to update openapi, except that it was not compatible with recent
versions of docutils, and that this would cause openapi's test suite to
fail to pass.

... So we ended up forking the openapi repository to update the
dependency to sphinx-mdinclude locally, and this is what we've been
using since last summer. And things were good again.

But things are even better when they go upstream [citation needed]. We
also filed the issue for docutils compatibility in sphinx-mdinclude [1].
It was fixed (thanks!). We submitted a PR to have openapi switch to
sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag
was created.

Now at last, we can switch back to the upstream version of openapi!
[And the build system lived happily ever after.]

[0]: GHSA-fw3v-x4f2-v673
[1]: omnilib/sphinx-mdinclude#8
[2]: sphinx-contrib/openapi#127

I did _not_ run `make -C Documentation update-requirements`, because the
resulting changes seemed to break the Netlify preview [3]. I stuck to
openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi.

[3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
joestringer pushed a commit to cilium/cilium that referenced this issue Feb 4, 2023
Once upon a time, Cilium docs used the openapi Sphinx add-on to generate
its API reference based on the code. And things were good.

One day, Dependabot raised a security alert, stating that Mistune v2.0.2
was vulnerable to catastrophic backtracking [0] - this is a regex
parsing thing. Mistune was a dependency to m2r, an add-on to parse
Markdown in Sphinx, which in turn was a dependency to openapi.

The easy path would have been to update m2r to use the latest, fixed
Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and
also it was no longer in development.

There was a fork, m2r2, which had little activity, and would avoid the
security issue by very simply pinning the Mistune version to 0.8.4
(which would either fail to build Cilium's reference correctly, or bring
some incompatibilities with other dependencies, at this point the
narrator does not remember for sure).

There was a fork of the fork, sphinx-mdinclude. We could use that
project to update openapi, except that it was not compatible with recent
versions of docutils, and that this would cause openapi's test suite to
fail to pass.

... So we ended up forking the openapi repository to update the
dependency to sphinx-mdinclude locally, and this is what we've been
using since last summer. And things were good again.

But things are even better when they go upstream [citation needed]. We
also filed the issue for docutils compatibility in sphinx-mdinclude [1].
It was fixed (thanks!). We submitted a PR to have openapi switch to
sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag
was created.

Now at last, we can switch back to the upstream version of openapi!
[And the build system lived happily ever after.]

[0]: GHSA-fw3v-x4f2-v673
[1]: omnilib/sphinx-mdinclude#8
[2]: sphinx-contrib/openapi#127

I did _not_ run `make -C Documentation update-requirements`, because the
resulting changes seemed to break the Netlify preview [3]. I stuck to
openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi.

[3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
aanm pushed a commit to pchaigno/cilium that referenced this issue Feb 13, 2023
[ upstream commit e4889d7 ]

Once upon a time, Cilium docs used the openapi Sphinx add-on to generate
its API reference based on the code. And things were good.

One day, Dependabot raised a security alert, stating that Mistune v2.0.2
was vulnerable to catastrophic backtracking [0] - this is a regex
parsing thing. Mistune was a dependency to m2r, an add-on to parse
Markdown in Sphinx, which in turn was a dependency to openapi.

The easy path would have been to update m2r to use the latest, fixed
Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and
also it was no longer in development.

There was a fork, m2r2, which had little activity, and would avoid the
security issue by very simply pinning the Mistune version to 0.8.4
(which would either fail to build Cilium's reference correctly, or bring
some incompatibilities with other dependencies, at this point the
narrator does not remember for sure).

There was a fork of the fork, sphinx-mdinclude. We could use that
project to update openapi, except that it was not compatible with recent
versions of docutils, and that this would cause openapi's test suite to
fail to pass.

... So we ended up forking the openapi repository to update the
dependency to sphinx-mdinclude locally, and this is what we've been
using since last summer. And things were good again.

But things are even better when they go upstream [citation needed]. We
also filed the issue for docutils compatibility in sphinx-mdinclude [1].
It was fixed (thanks!). We submitted a PR to have openapi switch to
sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag
was created.

Now at last, we can switch back to the upstream version of openapi!
[And the build system lived happily ever after.]

[0]: GHSA-fw3v-x4f2-v673
[1]: omnilib/sphinx-mdinclude#8
[2]: sphinx-contrib/openapi#127

I did _not_ run `make -C Documentation update-requirements`, because the
resulting changes seemed to break the Netlify preview [3]. I stuck to
openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi.

[3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Paul Chaignon <paul@cilium.io>
aanm pushed a commit to cilium/cilium that referenced this issue Feb 13, 2023
[ upstream commit e4889d7 ]

Once upon a time, Cilium docs used the openapi Sphinx add-on to generate
its API reference based on the code. And things were good.

One day, Dependabot raised a security alert, stating that Mistune v2.0.2
was vulnerable to catastrophic backtracking [0] - this is a regex
parsing thing. Mistune was a dependency to m2r, an add-on to parse
Markdown in Sphinx, which in turn was a dependency to openapi.

The easy path would have been to update m2r to use the latest, fixed
Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and
also it was no longer in development.

There was a fork, m2r2, which had little activity, and would avoid the
security issue by very simply pinning the Mistune version to 0.8.4
(which would either fail to build Cilium's reference correctly, or bring
some incompatibilities with other dependencies, at this point the
narrator does not remember for sure).

There was a fork of the fork, sphinx-mdinclude. We could use that
project to update openapi, except that it was not compatible with recent
versions of docutils, and that this would cause openapi's test suite to
fail to pass.

... So we ended up forking the openapi repository to update the
dependency to sphinx-mdinclude locally, and this is what we've been
using since last summer. And things were good again.

But things are even better when they go upstream [citation needed]. We
also filed the issue for docutils compatibility in sphinx-mdinclude [1].
It was fixed (thanks!). We submitted a PR to have openapi switch to
sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag
was created.

Now at last, we can switch back to the upstream version of openapi!
[And the build system lived happily ever after.]

[0]: GHSA-fw3v-x4f2-v673
[1]: omnilib/sphinx-mdinclude#8
[2]: sphinx-contrib/openapi#127

I did _not_ run `make -C Documentation update-requirements`, because the
resulting changes seemed to break the Netlify preview [3]. I stuck to
openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi.

[3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Paul Chaignon <paul@cilium.io>
pchaigno pushed a commit to pchaigno/cilium that referenced this issue Feb 15, 2023
[ upstream commit e4889d7 ]

Once upon a time, Cilium docs used the openapi Sphinx add-on to generate
its API reference based on the code. And things were good.

One day, Dependabot raised a security alert, stating that Mistune v2.0.2
was vulnerable to catastrophic backtracking [0] - this is a regex
parsing thing. Mistune was a dependency to m2r, an add-on to parse
Markdown in Sphinx, which in turn was a dependency to openapi.

The easy path would have been to update m2r to use the latest, fixed
Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and
also it was no longer in development.

There was a fork, m2r2, which had little activity, and would avoid the
security issue by very simply pinning the Mistune version to 0.8.4
(which would either fail to build Cilium's reference correctly, or bring
some incompatibilities with other dependencies, at this point the
narrator does not remember for sure).

There was a fork of the fork, sphinx-mdinclude. We could use that
project to update openapi, except that it was not compatible with recent
versions of docutils, and that this would cause openapi's test suite to
fail to pass.

... So we ended up forking the openapi repository to update the
dependency to sphinx-mdinclude locally, and this is what we've been
using since last summer. And things were good again.

But things are even better when they go upstream [citation needed]. We
also filed the issue for docutils compatibility in sphinx-mdinclude [1].
It was fixed (thanks!). We submitted a PR to have openapi switch to
sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag
was created.

Now at last, we can switch back to the upstream version of openapi!
[And the build system lived happily ever after.]

[0]: GHSA-fw3v-x4f2-v673
[1]: omnilib/sphinx-mdinclude#8
[2]: sphinx-contrib/openapi#127

I did _not_ run `make -C Documentation update-requirements`, because the
resulting changes seemed to break the Netlify preview [3]. I stuck to
openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi.

[3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Paul Chaignon <paul@cilium.io>
pchaigno pushed a commit to cilium/cilium that referenced this issue Feb 16, 2023
[ upstream commit e4889d7 ]

Once upon a time, Cilium docs used the openapi Sphinx add-on to generate
its API reference based on the code. And things were good.

One day, Dependabot raised a security alert, stating that Mistune v2.0.2
was vulnerable to catastrophic backtracking [0] - this is a regex
parsing thing. Mistune was a dependency to m2r, an add-on to parse
Markdown in Sphinx, which in turn was a dependency to openapi.

The easy path would have been to update m2r to use the latest, fixed
Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and
also it was no longer in development.

There was a fork, m2r2, which had little activity, and would avoid the
security issue by very simply pinning the Mistune version to 0.8.4
(which would either fail to build Cilium's reference correctly, or bring
some incompatibilities with other dependencies, at this point the
narrator does not remember for sure).

There was a fork of the fork, sphinx-mdinclude. We could use that
project to update openapi, except that it was not compatible with recent
versions of docutils, and that this would cause openapi's test suite to
fail to pass.

... So we ended up forking the openapi repository to update the
dependency to sphinx-mdinclude locally, and this is what we've been
using since last summer. And things were good again.

But things are even better when they go upstream [citation needed]. We
also filed the issue for docutils compatibility in sphinx-mdinclude [1].
It was fixed (thanks!). We submitted a PR to have openapi switch to
sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag
was created.

Now at last, we can switch back to the upstream version of openapi!
[And the build system lived happily ever after.]

[0]: GHSA-fw3v-x4f2-v673
[1]: omnilib/sphinx-mdinclude#8
[2]: sphinx-contrib/openapi#127

I did _not_ run `make -C Documentation update-requirements`, because the
resulting changes seemed to break the Netlify preview [3]. I stuck to
openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi.

[3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Paul Chaignon <paul@cilium.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants