Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KeyError: 'domain' exception being thrown from PropagateDescDomain.run #9322

Closed
bneradt opened this issue Jun 11, 2021 · 1 comment
Closed

KeyError: 'domain' exception being thrown from PropagateDescDomain.run #9322

bneradt opened this issue Jun 11, 2021 · 1 comment
Labels
type:bug
Milestone
4.0.3

Comments

@bneradt
Copy link

bneradt commented Jun 11, 2021

Describe the bug
Apache Traffic Server docs are built using Sphinx. We noticed that when we upgraded to the 4.x release the build breaks with the following output:

# Sphinx version: 4.0.2
# Python version: 3.8.3 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 3.0.1
# Last messages:
#   writing output... [  2%] admin-guide/configuration/transparent-proxy/wccp-service-config.en
#   writing output... [  2%] admin-guide/configuring-traffic-server.en
#   writing output... [  2%] admin-guide/files/cache.config.en
#   writing output... [  3%] admin-guide/files/hosting.config.en
#   writing output... [  3%] admin-guide/files/index.en
#   writing output... [  3%] admin-guide/files/ip_allow.yaml.en
#   writing output... [  3%] admin-guide/files/logging.yaml.en
#   writing output... [  3%] admin-guide/files/parent.config.en
#   writing output... [  4%] admin-guide/files/plugin.config.en
#   writing output... [  4%] admin-guide/files/records.config.en
# Loaded extensions:
#   sphinx.ext.mathjax (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/alabaster/__init__.py
#   sphinx.ext.graphviz (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/graphviz.py
#   sphinx.ext.intersphinx (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/intersphinx.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.todo (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/todo.py
#   sphinx.ext.coverage (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/coverage.py
#   sphinx.ext.viewcode (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/viewcode.py
#   sphinxcontrib.plantuml (unknown version) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinxcontrib/plantuml.py
#   traffic-server (unknown version) from /var/tmp/trafficserver/doc/ext/traffic-server.py
#   sphinx.ext.imgmath (4.0.2) from /root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/ext/imgmath.py
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/application.py", line 350, in build
    self.builder.build_update()
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 292, in build_update
    self.build(to_build,
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 356, in build
    self.write(docnames, list(updated_docnames), method)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 530, in write
    self._write_serial(sorted(docnames))
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 537, in _write_serial
    doctree = self.env.get_and_resolve_doctree(docname, self)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/environment/__init__.py", line 527, in get_and_resolve_doctree
    self.apply_post_transforms(doctree, docname)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/environment/__init__.py", line 573, in apply_post_transforms
    transformer.apply_transforms()
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/transforms/__init__.py", line 86, in apply_transforms
    super().apply_transforms()
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/transforms/post_transforms/__init__.py", line 42, in apply
    self.run(**kwargs)
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/sphinx/transforms/post_transforms/__init__.py", line 252, in run
    node['classes'].append(node.parent['domain'])
  File "/root/.local/share/virtualenvs/doc-2zq9zEzg/lib/python3.8/site-packages/docutils/nodes.py", line 625, in __getitem__
    return self.attributes[key]
KeyError: 'domain'

If we pin the Sphinx package to 3.5.4, however, the docs build fine.

To Reproduce
Steps to reproduce the behavior:

docker pull controller.trafficserver.org/ats/centos:8
docker run --name build_ats_docs --init --cap-add=SYS_PTRACE --network=host -d -t controller.trafficserver.org/ats/centos:8
docker exec -it build_ats_docs /bin/bash

cd /var/tmp
git clone https://github.com/apache/trafficserver.git
cd trafficserver

# Checkout the following commit as a Sphinx package pin workaround will soon be merged.
git checkout 1177cc7056294a3b1aaa506456a76cdf6dcc9b52

source /opt/rh/gcc-toolset-9/enable
yum install -y java
cd doc

#-------------------------------------------------------
# Note: at the present time this will install Sphinx 4.0.2:
#-------------------------------------------------------
pipenv install


pipenv shell

cd ..
autoreconf -fi
./configure --enable-docs
cd doc
make html

Some environment information:

(doc) [root@docker-desktop doc]# pip freeze
alabaster==0.7.12
Babel==2.9.1
certifi==2021.5.30
chardet==4.0.0
click==8.0.1
docutils==0.16
idna==2.10
imagesize==1.2.0
Jinja2==3.0.1
lxml==4.6.3
MarkupSafe==2.0.1
packaging==20.9
polib==1.1.1
Pygments==2.9.0
pyparsing==2.4.7
pytz==2021.1
requests==2.25.1
snowballstemmer==2.1.0
Sphinx==4.0.2
sphinx-intl==2.0.1
sphinx-rtd-theme==0.5.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-plantuml==0.21
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
urllib3==1.26.5
(doc) [root@docker-desktop doc]# python --version
Python 3.8.3

Expected behavior
If the Sphinx package is pinned to 3.5.4, the build succeeds. Instead, with 4.0.2, it fails with the 'domain' KeyError exception.

Your project
https://github.com/apache/trafficserver/

(See the above clone and checkout git commands to reproduce this.)

Screenshots
Not applicable.

Environment info
The above replay steps show how to reproduce the environment I'm seeing this with using docker.

  • OS: CentOS 8, docker image: controller.trafficserver.org/ats/centos:8
  • Python version: 3.8.3
  • Sphinx version: 4.0.2
  • Sphinx extensions:
  • Extra tools:

Additional context

From Traffic Server's perspective, this is being tracked via the following Apache Traffic Server issue which will soon be worked around by pinning our Sphinx release to 3.x:
@bneradt bneradt mentioned this issue Jun 11, 2021
Closed
@tk0miya
Copy link
Member

tk0miya commented Jun 13, 2021

The current implementation of PropagateDescDomain expects a parent of the desc_signature node is a desc node and it has "domain" attribute. But it's not a restriction. So I'll fix the implementation later.

tk0miya added a commit to tk0miya/sphinx that referenced this issue Jun 13, 2021
@tk0miya
Fix sphinx-doc#9322: KeyError is raised on PropagateDescDomain transform
2150923 
PropageteDescDomain applies the domain name from the "domain" attribute
of parent node (desc node) to the desc_signature node.  The structure
has longly generated by ObjectDescription.  But it must not be a new
rule.

This allows to build document that contains non standard doctree.
tk0miya added a commit to tk0miya/sphinx that referenced this issue Jun 13, 2021
@tk0miya
Fix sphinx-doc#9322: KeyError is raised on PropagateDescDomain transform
2de7c35 
PropageteDescDomain applies the domain name from the "domain" attribute
of parent node (desc node) to the desc_signature node.  The structure
has longly generated by ObjectDescription.  But it must not be a new
rule.

This allows to build document that contains non standard doctree.
@tk0miya tk0miya mentioned this issue Jun 13, 2021
Merged
@tk0miya tk0miya added this to the 4.0.3 milestone Jun 13, 2021
@bneradt bneradt mentioned this issue Jun 14, 2021
Closed
@bneradt bneradt mentioned this issue Jun 23, 2021
Closed
tk0miya added a commit that referenced this issue Jun 29, 2021
@tk0miya
Merge pull request #9334 from tk0miya/9322_KeyError_from_PropageteDes…
723ad78 
…cDomain

Fix #9322: KeyError is raised on PropagateDescDomain transform
@tk0miya tk0miya closed this as completed Jun 29, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
type:bug
Projects
None yet
Milestone
4.0.3
Development

No branches or pull requests
2 participants
@tk0miya @bneradt