The objective of this tutorial is to create a more comprehensive extension than that created in helloworld
. Whereas that guide just covered writing a custom directive
, this guide adds multiple directives, along with custom nodes, additional config values and custom event handlers. To this end, we will cover a todo
extension that adds capabilities to include todo entries in the documentation, and to collect these in a central place. This is similar the sphinxext.todo
extension distributed with Sphinx.
Note
To understand the design of this extension, refer to important-objects
and build-phases
.
We want the extension to add the following to Sphinx:
- A
todo
directive, containing some content that is marked with "TODO" and only shown in the output if a new config value is set. Todo entries should not be in the output by default. - A
todolist
directive that creates a list of all todo entries throughout the documentation.
For that, we will need to add the following elements to Sphinx:
- New directives, called
todo
andtodolist
. - New document tree nodes to represent these directives, conventionally also called
todo
andtodolist
. We wouldn't need new nodes if the new directives only produced some content representable by existing nodes. - A new config value
todo_include_todos
(config value names should start with the extension name, in order to stay unique) that controls whether todo entries make it into the output. - New event handlers: one for the
doctree-resolved
event, to replace the todo and todolist nodes, one forenv-merge-info
to merge intermediate results from parallel builds, and one forenv-purge-doc
(the reason for that will be covered later).
As with helloworld
, we will not be distributing this plugin via PyPI so once again we need a Sphinx project to call this from. You can use an existing project or create a new one using sphinx-quickstart
.
We assume you are using separate source (source
) and build (build
) folders. Your extension file could be in any folder of your project. In our case, let's do the following:
- Create an
_ext
folder insource
- Create a new Python file in the
_ext
folder calledtodo.py
Here is an example of the folder structure you might obtain:
└── source
├── _ext
│ └── todo.py
├── _static
├── conf.py
├── somefolder
├── index.rst
├── somefile.rst
└── someotherfile.rst
Open todo.py
and paste the following code in it, all of which we will explain in detail shortly:
examples/todo.py
This is far more extensive extension than the one detailed in helloworld
, however, we will will look at each piece step-by-step to explain what's happening.
The node classes
Let's start with the node classes:
examples/todo.py
Node classes usually don't have to do anything except inherit from the standard docutils classes defined in docutils.nodes
. todo
inherits from Admonition
because it should be handled like a note or warning, todolist
is just a "general" node.
Note
Many extensions will not have to create their own node classes and work fine with the nodes already provided by docutils and Sphinx
<nodes>
.
Attention
It is important to know that while you can extend Sphinx without leaving your conf.py
, if you declare an inherited node right there, you'll hit an unobvious :pyPickleError
. So if something goes wrong, please make sure that you put inherited nodes into a separate Python module.
For more details, see:
The directive classes
A directive class is a class deriving usually from docutils.parsers.rst.Directive
. The directive interface is also covered in detail in the docutils documentation; the important thing is that the class should have attributes that configure the allowed markup, and a run
method that returns a list of nodes.
Looking first at the TodolistDirective
directive:
examples/todo.py
It's very simple, creating and returning an instance of our todolist
node class. The TodolistDirective
directive itself has neither content nor arguments that need to be handled. That brings us to the TodoDirective
directive:
examples/todo.py
Several important things are covered here. First, as you can see, we're now subclassing the ~sphinx.util.docutils.SphinxDirective
helper class instead of the usual ~docutils.parsers.rst.Directive
class. This gives us access to the build environment instance <important-objects>
using the self.env
property. Without this, we'd have to use the rather convoluted self.state.document.settings.env
. Then, to act as a link target (from TodolistDirective
), the TodoDirective
directive needs to return a target node in addition to the todo
node. The target ID (in HTML, this will be the anchor name) is generated by using env.new_serialno
which returns a new unique integer on each call and therefore leads to unique target names. The target node is instantiated without any text (the first two arguments).
On creating admonition node, the content body of the directive are parsed using self.state.nested_parse
. The first argument gives the content body, and the second one gives content offset. The third argument gives the parent node of parsed result, in our case the todo
node. Following this, the todo
node is added to the environment. This is needed to be able to create a list of all todo entries throughout the documentation, in the place where the author puts a todolist
directive. For this case, the environment attribute todo_all_todos
is used (again, the name should be unique, so it is prefixed by the extension name). It does not exist when a new environment is created, so the directive must check and create it if necessary. Various information about the todo entry's location are stored along with a copy of the node.
In the last line, the nodes that should be put into the doctree are returned: the target node and the admonition node.
The node structure that the directive returns looks like this:
+--------------------+
| target node |
+--------------------+
+--------------------+
| todo node |
+--------------------+
\__+--------------------+
| admonition title |
+--------------------+
| paragraph |
+--------------------+
| ... |
+--------------------+
The event handlers
Event handlers are one of Sphinx's most powerful features, providing a way to do hook into any part of the documentation process. There are many events provided by Sphinx itself, as detailed in the API guide <events>
, and we're going to use a subset of them here.
Let's look at the event handlers used in the above example. First, the one for the env-purge-doc
event:
examples/todo.py
Since we store information from source files in the environment, which is persistent, it may become out of date when the source file changes. Therefore, before each source file is read, the environment's records of it are cleared, and the env-purge-doc
event gives extensions a chance to do the same. Here we clear out all todos whose docname matches the given one from the todo_all_todos
list. If there are todos left in the document, they will be added again during parsing.
The next handler, for the env-merge-info
event, is used during parallel builds. As during parallel builds all threads have their own env
, there's multiple todo_all_todos
lists that need to be merged:
examples/todo.py
The other handler belongs to the doctree-resolved
event:
examples/todo.py
The doctree-resolved
event is emitted at the end of phase 3
(resolving) <build-phases>
and allows custom resolving to be done. The handler we have written for this event is a bit more involved. If the todo_include_todos
config value (which we'll describe shortly) is false, all todo
and todolist
nodes are removed from the documents. If not, todo
nodes just stay where and how they are. todolist
nodes are replaced by a list of todo entries, complete with backlinks to the location where they come from. The list items are composed of the nodes from the todo
entry and docutils nodes created on the fly: a paragraph for each entry, containing text that gives the location, and a link (reference node containing an italic node) with the backreference. The reference URI is built by sphinx.builders.Builder.get_relative_uri
which creates a suitable URI depending on the used builder, and appending the todo node's (the target's) ID as the anchor name.
The setup
function
sphinx.application
As noted previously <helloworld>
, the setup
function is a requirement and is used to plug directives into Sphinx. However, we also use it to hook up the other parts of our extension. Let's look at our setup
function:
examples/todo.py
The calls in this function refer to the classes and functions we added earlier. What the individual calls do is the following:
~Sphinx.add_config_value
lets Sphinx know that it should recognize the new config valuetodo_include_todos
, whose default value should beFalse
(this also tells Sphinx that it is a boolean value).If the third argument was
'html'
, HTML documents would be full rebuild if the config value changed its value. This is needed for config values that influence reading (buildphase 1 (reading) <build-phases>
).~Sphinx.add_node
adds a new node class to the build system. It also can specify visitor functions for each supported output format. These visitor functions are needed when the new nodes stay untilphase 4 (writing) <build-phases>
. Since thetodolist
node is always replaced inphase 3 (resolving) <build-phases>
, it doesn't need any.~Sphinx.add_directive
adds a new directive, given by name and class.- Finally,
~Sphinx.connect
adds an event handler to the event whose name is given by the first argument. The event handler function is called with several arguments which are documented with the event.
With this, our extension is complete.
As before, we need to enable the extension by declaring it in our conf.py
file. There are two steps necessary here:
- Add the
_ext
directory to the Python path usingsys.path.append
. This should be placed at the top of the file. - Update or create the
extensions
list and add the extension file name to the list
In addition, we may wish to set the todo_include_todos
config value. As noted above, this defaults to False
but we can set it explicitly.
For example:
import os
import sys
sys.path.append(os.path.abspath("./_ext"))
extensions = ['todo']
todo_include_todos = False
You can now use the extension throughout your project. For example:
Hello, world
============
.. toctree::
somefile.rst
someotherfile.rst
Hello world. Below is the list of TODOs.
.. todolist::
foo
===
Some intro text here...
.. todo:: Fix this
bar
===
Some more text here...
.. todo:: Fix that
Because we have configured todo_include_todos
to False
, we won't actually see anything rendered for the todo
and todolist
directives. However, if we toggle this to true, we will see the output described previously.
For more information, refer to the docutils documentation and /extdev/index
.