[WIP?] C, add compatibility flag for parsing some pre-v3 input

[jakobandersen]

The C domain got much stricter in its parsing in version 3 and added additional directives and roles to provide better markup. This PR adds the configuration variable c_allow_pre_v3. When set to True the type directive and role will try to convert some previously recognized input to the v3 equivalent.

Feature or Bugfix

• Feature

Detail

• Fixes C domain changes of Sphinx 3 prevent to write doc compatible with Sphinx 2 and Sphinx 3 #7899

[jakobandersen]

@vstinner, I believe this should take care of most cases that can not be changed to work in both v2 and v3. Let me know if there are more cases.

[vstinner]

In short, the PR works as expected, thanks! I just had to remove -W option from sphinx-build command used by Python.

---

I built the Python documentation with this PR.

make check SPHINXOPTS="-q -W -j4" succeed.

make suspicious SPHINXOPTS="-q -W -j4" fails with:

Doc/extending/extending.rst.rst:160:RemovedInSphinx40Warning: Pre-v3 C type role ':c:type:`PyErr_*`' converted to ':c:expr:`PyErr_*`'.
The original parsing error was:
Invalid C declaration: Expected end of definition. [error at 6]
  PyErr_*
  ------^

That's because sphinx-build is run with -W to treat warnings as errors.

Without -W, make suspicious SPHINXOPTS="-q -j4" SPHINXERRORHANDLING="" (check for suspicious syntax) and make html SPHINXOPTS="-q -j4" SPHINXERRORHANDLING="" (render as HTML) commands succeed (exit code 0), but log tons of warnings (which is acceptable for now).

[jakobandersen]

There should now be a c_warn_on_allowed_pre_v3 option you can set to False to disable these warnings. The behaviour is still deprecated though.
Would v4 be late enough for removal, or should we bump it to v5?

[vstinner]

FYI the issue to track the Sphinx 3 compatibility in Python is: https://bugs.python.org/issue40204

[vstinner]

There should now be a c_warn_on_allowed_pre_v3 option you can set to False to disable these warnings. The behaviour is still deprecated though. Would v4 be late enough for removal, or should we bump it to v5?

FYI in Python, we try to wait at least 2 years between a feature is deprecated and its removal. I have no idea when v4 and v5 are planned. All I know is that so far, I'm only aware of Fedora Rawhide which ugraded to Sphinx 3, and there are multiple packages broken by this upgrade:
https://bugzilla.redhat.com/showdependencytree.cgi?id=SPHINX3&hide_resolved=1

[jakobandersen]

If I remember correctly the major versions are released once per year, so v4 would be in spring 2021.

[vstinner]

What's the next step for this PEP?

[jakobandersen]

Basically it can be merged (after I add docs and changelog entry), but how does it work? Are there more cases where we should try to do something?
I tried compiling the Python docs, but there were other warnings that prevented me from quickly seeing if every case I expected were handled.

[vstinner]

but how does it work?

As I wrote, this PR makes possible to build the Python documentation. For the warnings, I don't have the bandwidth to go through the huge pile of warnings. Some are real issues in the doc, some are deprecation warnings that nobody tried to address, etc.

[jakobandersen]

Great, and yes, there were indeed a lot of warnings. Some still related to the C domain changes, but should be fixed on the Python side, and can be in a v2 compatible manner.
I'll get this merged when as soon as I get the docs updated.

[jakobandersen]

This should be in the upcomming v3.2, and is scheduled to be removed in v5.0.