Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unknown node: pending_xref_condition #9240

Closed
RuRo opened this issue May 16, 2021 · 7 comments
Closed

Unknown node: pending_xref_condition #9240

RuRo opened this issue May 16, 2021 · 7 comments

Comments

@RuRo
Copy link
Contributor

RuRo commented May 16, 2021

To Reproduce

python -m venv .venv
. .venv/bin/activate
pip install sphinx==4.0.1 sphinx-qt-documentation==0.3 PyQt5==5.15.4
mkdir docs

foo.py:

from PyQt5 import QtGui

def bar() -> QtGui.QIcon:
    pass

docs/conf.py:

import os
import sys

sys.path.insert(0, os.path.abspath("./"))

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.intersphinx",
    "sphinx_qt_documentation",
]
intersphinx_mapping = {
    "PyQt5": ("https://riverbankcomputing.com/static/Docs/PyQt5/", None),
}
python_use_unqualified_type_names = True

docs/index.rst:

test
====

.. automodule:: foo
   :members:
   :undoc-members:

Run:

python -m sphinx -b dirhtml docs .out

Result

# Sphinx version: 4.0.1
# Python version: 3.9.4 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 2.11.3
# Last messages:
#   
#   looking for now-outdated files...
#   none found
#   pickling environment...
#   done
#   checking consistency...
#   done
#   preparing documents...
#   done
#   writing output... [100%] index
# Loaded extensions:
#   sphinx.ext.mathjax (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (1.0.3) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.4) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/alabaster/__init__.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.intersphinx (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/intersphinx.py
#   sphinx_qt_documentation (0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx_qt_documentation/__init__.py
Traceback (most recent call last):
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/application.py", line 350, in build
    self.builder.build_update()
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 292, in build_update
    self.build(to_build,
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 356, in build
    self.write(docnames, list(updated_docnames), method)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 530, in write
    self._write_serial(sorted(docnames))
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 540, in _write_serial
    self.write_doc(docname, doctree)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/html/__init__.py", line 615, in write_doc
    self.docwriter.write(doctree, destination)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/writers/html.py", line 70, in translate
    self.document.walkabout(visitor)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  [Previous line repeated 3 more times]
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 219, in walkabout
    visitor.dispatch_visit(self)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/util/docutils.py", line 472, in dispatch_visit
    super().dispatch_visit(node)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 2021, in dispatch_visit
    return method(node)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/writers/html5.py", line 799, in unknown_visit
    raise NotImplementedError('Unknown node: ' + node.__class__.__name__)
NotImplementedError: Unknown node: pending_xref_condition

Environment info

  • OS: Linux
  • Python version: 3.9.4
  • Sphinx version: 4.0.1
  • Sphinx extensions: sphinx.ext.autodoc, sphinx.ext.intersphinx, sphinx_qt_documentation

Additional context

This is similar to the issue that was fixed in #8996. I think, that I've tracked the issue to the sphinx-qt-documentation plugin. This plugin connects itself to missing-reference events like this

def setup(app: Sphinx) -> Dict[str, Any]:
    ... # snip
    app.connect("missing-reference", missing_reference)
    ... # snip

however, their missing_reference implementation doesn't have the

    content = find_pending_xref_condition(node, 'resolved')
    if content:
        contnode = content.children[0]  # type: ignore

snippet, which eventually leads to the above Unknown node: pending_xref_condition error.

I could submit this as a bug report to the author of sphinx-qt-documentation, but I wanted to make sure, that this behaviour is intended. It seems to me like a potential antipattern. For example, the above snippet (with minor variations) is already duplicated internally in

  • sphinx/domains/python.py:PythonDomain.resolve_xref
  • sphinx/domains/python.py:PythonDomain.resolve_any_xref
  • sphinx/domains/python.py:builtin_resolver
  • sphinx/ext/intersphinx.py:missing_reference

And any plugin that connects to the missing-reference event must also now add this snippet (which is complicated by the fact that find_pending_xref_condition doesn't seem to be available in sphinx<4). Do you think that maybe the contnode value should be resolved when the missing-reference event is created in sphinx/transforms/post_transforms/__init__.py?

@tk0miya
Copy link
Member

tk0miya commented May 16, 2021

Thank you for reporting. I guess Sphinx's post processing has a bug when 3rd party extension resolves a missing-reference.

Could you check this patch works fine?

diff --git a/sphinx/transforms/post_transforms/__init__.py b/sphinx/transforms/post_transforms/__init__.py
index e2899d994..54cab4ed6 100644
--- a/sphinx/transforms/post_transforms/__init__.py
+++ b/sphinx/transforms/post_transforms/__init__.py
@@ -96,10 +96,18 @@ class ReferencesResolver(SphinxPostTransform):
                     newnode = self.app.emit_firstresult('missing-reference', self.env,
                                                         node, contnode,
                                                         allowed_exceptions=(NoUri,))
-                    # still not found? warn if node wishes to be warned about or
-                    # we are in nit-picky mode
                     if newnode is None:
+                        # still not found? warn if node wishes to be warned about or
+                        # we are in nit-picky mode
                         self.warn_missing_reference(refdoc, typ, target, node, domain)
+                    elif isinstance(newnode[0], addnodes.pending_xref_condition):
+                        matched = find_pending_xref_condition(node, "*")
+                        if matched:
+                            newnode = matched[0]
+                        else:
+                            logger.warning(__('Could not determine the fallback text for the '
+                                              'cross-reference. Might be a bug.'),
+                                           location=node)
             except NoUri:
                 newnode = None

(I'll try this tomorrow. But I need to build the environment to build the example project. I don't know PyQt at all...)

@RuRo
Copy link
Contributor Author

RuRo commented May 16, 2021

Thank you for reporting. I guess Sphinx's post processing has a bug when 3rd party extension resolves a missing-reference.

Could you check this patch works fine?

Not quite. It removes the exception, but the PyQt5.QtGui.QIcon return type is still rendered including with the full module name, and it doesn't link to the Qt docs.

I was thinking of something more along the lines of

--- a/sphinx/transforms/post_transforms/__init__.py
+++ b/sphinx/transforms/post_transforms/__init__.py
@@ -69,11 +69,15 @@
 
     default_priority = 10
 
     def run(self, **kwargs: Any) -> None:
         for node in self.document.traverse(addnodes.pending_xref):
-            contnode = cast(nodes.TextElement, node[0].deepcopy())
+            content = find_pending_xref_condition(node, 'resolved')
+            if content:
+                contnode = content.children[0]  # type: ignore
+            else:
+                contnode = cast(nodes.TextElement, node[0].deepcopy())
             newnode = None
 
             typ = node['reftype']
             target = node['reftarget']
             refdoc = node.get('refdoc', self.env.docname)

(although you might need to somehow handle the case, where content has multiple children, I am not 100% sure)

The above patch fixes the error and produces the desired results for the minimal test case, that I have provided, but I am not sure, if it plays well with other components of the reference resolution process.


With a little refactoring you should also be able to remove the duplicated code fragments in the following functions:

  • sphinx/domains/python.py:PythonDomain.resolve_xref
  • sphinx/domains/python.py:PythonDomain.resolve_any_xref
  • sphinx/domains/python.py:builtin_resolver
  • sphinx/ext/intersphinx.py:missing_reference

All these functions are called from ReferenceResolver.run either directly:

  • PythonDomain.resolve_xref by domain.resolve_xref
  • PythonDomain.resolve_any_xref from self.resolve_anyref by domain.resolve_any_xref

or via connecting to the missing-reference event:

  • domains/python.py:builtin_resolver by app.connect('missing-reference', builtin_resolver, priority=900)
  • ext/intersphinx.py:missing_reference by app.connect('missing-reference', missing_reference))

(I'll try this tomorrow. But I need to build the environment to build the example project. I don't know PyQt at all...)

You shouldn't need any PyQt knowledge for this. The reason why Qt is involved here is that Qt doesn't have a full Python documentation, because it's a C++ wrapper. So the sphinx-qt-documentation plugin just makes it possible to automatically link from a python type to documentation of the equivalent C++ type. For example, the PyQt5.QtGui.QIcon return type in foo.py should link to the C++ docs for the QIcon class.

tk0miya added a commit to tk0miya/sphinx that referenced this issue May 17, 2021
… raised

Unknown node error for pending_xref_condition is raised if an extension
that does not support the node installs a missing-reference handler.
tk0miya added a commit to tk0miya/sphinx that referenced this issue May 17, 2021
… raised

Unknown node error for pending_xref_condition is raised if an extension
that does not support the node installs a missing-reference handler.
tk0miya added a commit to tk0miya/sphinx that referenced this issue May 17, 2021
… raised

Unknown node error for pending_xref_condition is raised if an extension
that does not support the node installs a missing-reference handler.
tk0miya added a commit to tk0miya/sphinx that referenced this issue May 17, 2021
… raised

Unknown node error for pending_xref_condition is raised if an extension
that does not support the node installs a missing-reference handler.
tk0miya added a commit to tk0miya/sphinx that referenced this issue May 17, 2021
… raised

Unknown node error for pending_xref_condition is raised if an extension
that does not support the node installs a missing-reference handler.
@tk0miya
Copy link
Member

tk0miya commented May 17, 2021

Thank you for confirmation. And it seems your proposal is better than mine. I posted #9246 to resolve this issue. Could you confirm this, please?

You shouldn't need any PyQt knowledge for this.

On my local, I can't install PyQt5. So I'd like to know the way to build the environment.

root@5de8942ddfb0:/# python
Python 3.8.5 (default, Aug  4 2020, 16:24:08)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5 import QtGui
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libGL.so.1: cannot open shared object file: No such file or directory

@RuRo
Copy link
Contributor Author

RuRo commented May 17, 2021

Could you confirm this, please?

I've tried the 9246 fix, and it seems to work fine for me.

ImportError: libGL.so.1: cannot open shared object file: No such file or directory

Ah, do you have a headless machine or something? Unfortunately, Qt has some binary requirements, that you can't install into the virtual environment (at least not easily). So you will have to install some packages to your actual system. Try installing python3-pyqt5 directly from apt-get or your package manager. This should also install all the binary dependencies, so the PyQt inside the virtual environment should now work.

On my local, I can't install PyQt5.

Why not? Do you mean that you don't have the admin privileges? Unfortunately, I don't think there's anything you can do in that case other than contacting your admin.

@tk0miya
Copy link
Member

tk0miya commented May 19, 2021

Thank you for confirming.

Try installing python3-pyqt5 directly from apt-get or your package manager.

Thank you for your wisdom. I confirm foo.py is running.

tk0miya added a commit that referenced this issue May 19, 2021
…condition

Fix #9240: Unknown node error for pending_xref_condition is raised
@tk0miya tk0miya reopened this May 19, 2021
@tk0miya
Copy link
Member

tk0miya commented May 19, 2021

Fixed in #9246. But keep opened for refactoring in 4.x branch.

@tk0miya tk0miya modified the milestones: 4.0.2, 4.1.0 May 19, 2021
tk0miya added a commit that referenced this issue May 19, 2021
tk0miya added a commit to tk0miya/sphinx that referenced this issue May 19, 2021
…doc#9240)

After sphinx-doc#9246, `find_pending_xref_conditions()` should be only called from
intended modules.  At present, the Python Domain is the only module to
call it intendedly.  Therefore, this removes the needless calls of the
utility function from "unintended" modules.
tk0miya added a commit to tk0miya/sphinx that referenced this issue May 22, 2021
…doc#9240)

After sphinx-doc#9246, `find_pending_xref_conditions()` should be only called from
intended modules.  At present, the Python Domain is the only module to
call it intendedly.  Therefore, this removes the needless calls of the
utility function from "unintended" modules.
tk0miya added a commit that referenced this issue May 22, 2021
…itions

refactor: reduce calls of find_pending_xref_conditions (refs: #9240)
@tk0miya
Copy link
Member

tk0miya commented Jun 5, 2021

Refactored by #9254. Closing.

@tk0miya tk0miya closed this as completed Jun 5, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants