Use autodoc for Python API reference

[brenthuisman]

Using the sphinx.ext.autodoc plugin to autogenerate Python Api documentation accomplishes the following:

• API documentation is never out of sync with the code.
• Less work: write documentation once!
• Consistency: help() and the docs will show the same documentation.

Requirements:

• sphinx.ext.autodoc requires an importable arbor module, so building the docs requires Arbor be built.
• Integrate current docs into docstrings. Format: Google style docstring + Python type annotations.
  • Python type annotation: https://docs.python.org/3/library/typing.html
  • Napoleon extension: Google style docstring (more brief)

Options:

• Module, class, member function or variable documentation can be inlined anywhere in the docs to facilitate deviation in the docs w.r.t docstrings if needed. Let's do this as little as possible (creates one more thing to track!).
• Autodoc shows all members by default. Can be tuned, this removes everything I consider noise for a full module documentation page:

.. automodule:: arbor
    :imported-members:
    :members:
    :undoc-members:
    :special-members: __init__
    :inherited-members:

[brenthuisman]

Regarding type signatures, things are hairy. An incomplete list of ingredients/options for generation the eventual html docs:

Python code:

• Python-sanctioned annotations
• List of PEPs about typing
• These can apparently live in separate stub files as per PEP 484, see also stubgen
• Pybind generates docstrings from the type info it can extract from the code. The quality of this is however in development. Pybind itself uses stubfiles (.pyi).
  • TODO: The format is Sphinx style docstrings [VERIFY, DUMP reference.rst].

Python docstrings:

• A Python docstring is a string placed in .__doc__.
  • Classes can rely on docstrings on its members or members can be documented in the class docstring.
  • Python comes with some features for formatting docstrings (used in e.g. help()), pydoc and components in docutils. Sphinx reuses some of this, but it's difficult to figure out what.
• Sphinx/reST style docstrings
• Numpy and Google style docstrings. This seems to convert those inputs to the Sphinx style docstring, which is then further processed.
• autodoc signatures.
• text_signature. Pybind does not use this.
• Bug: Type signature of @property not shown in help()

Signatures in docs is a duplication of information that you have to keep synchronized with API manually, and therefore has not my preference.

Pybind:

• Here module docstrings are created
• Here signatures are created
  • Find out how types is populated?
• Using stubgen, we can verify that Pybind generates the correct signatures (arbor.location.branch has one)
• Overloaded methods are not handled: Rework function signatures in docstrings of overloaded functions pybind/pybind11#2621

Sphinx:

• distinguishes type labels/field from type option/hint and formats them differently. I strongly prefer type options/hints. In the image: field/label up top, option/hint on the bottom
  As you see in the comment, I'm unable to generate a type option/hint from docstrings currently.

Various stubgens:

• pygenstub: does not generate type options/hints for member funcs.
• pybind11-stubgen generates stubs and copies docstrings, but fails to resolve some C++ types and does not seem to handle overloads; only shows func(*args, **kwargs) -> Any type of signatures in such case.
• mypy-stubgen. Does good job at type resolution (succeeds where pybind11-stubgen fails), but does not preserve docstrings. Misses some types still.
• The Sphinx Autodoc plugin sadly ignores type information provided in .pyi files.
  • They are actually valid Python code, so they could be used together with autodoc. However, the output is unordered, so there are many objects referred while not yet defined. Looked into MagicMock or other ways to autocorrect the order, no succes.

Problem:

• I want type option/hint output for any members, including readonly properties. Lower example in image demonstrates this.
• Sphinx/autodoc ignore type signatures on properties, which is how Pybind translates .def_readonly() etc. Bug in Python, so that seems why Sphinx can't generate the desired output, and pydoc doesn't show it (eg pydoc arbor.location.branch)
  • Using stubgen, we can verify that Pybind generates the correct signatures (arbor.location.branch has one). Using Pydoc, you can see that type information is not shown, indicating it never reaches Sphinx.
• Overloaded methods are not handled: Rework function signatures in docstrings of overloaded functions pybind/pybind11#2621

Summary:

At this time I'm not able to get the outputs (pervasive type options/hints) automatically generated, or documented in member docstrings. Potential alternatives that are not sufficient after all:

• Use mypy-stubgen generated stubs as importable modules for autdoc. This requires solving unconverted types, and does not work around the fact that docutils (?) does not show type info on properties.
• Document members in the class docstring. This duplicates info, and thus imposes a minor maintenance burden. This formats members yet another way, namely with a parameters label/field, but at least the type info is inline. Type info entered thus not available through help()/pydoc. Also, type info is not rendered as option/hint.
• Change nothing. Document members in the Sphinx sources as we do now. Is duplication, further removed from the code so an increased maintenance burden compared to previous option/hint. Type info entered thus not available through help()/pydoc.

Action:

• Blockers: Wait for fix to https://bugs.python.org/issue39125 (Python docutils supporting types for properties) and/or to Added support for reading .pyi files. sphinx-doc/sphinx#4824 (Sphinx support for .pyi)
  • sphinx-autoapi is not a solution for us because it only supports pure-Python.
• Do nothing. There is no way to get type option/hints with any alternative.

[brenthuisman]

##An option in the meantime:

In an earlier PR #1176, I wrote a script that hooks into Sphinx to dump the reST generated by autodoc to disk.

We could generate the autodoc reST out of band, demo, and then add in the type annotation by hand. For instance, arbor.location as generated from current docstrings:

.. py:class:: location(self: arbor.location, branch: int, pos: float) -> None
   :noindex:
   :module: arbor

   A location on a cable cell.

   A location on :attr:`branch`, where :attr:`pos`, in the range ``0 ≤ pos ≤ 1``,
   gives the relative position
   between the proximal and distal ends of the branch. The position is in terms
   of branch path length, so for example, on a branch of path length 100 μm ``pos=0.2``
   corresponds to 20 μm and 80 μm from the proximal and distal ends of the
   branch respectively.


   .. py:method:: location.branch
      :noindex:
      :module: arbor
      :property:

      The id of the branch.


   .. py:method:: location.pos
      :noindex:
      :module: arbor
      :property:

      The relative position on the branch (∈ [0.,1.], where 0. means proximal and 1. distal).

Adding in the type annotations by hand for the getters isn't much work:

.. py:class:: location(self: arbor.location, branch: int, pos: float) -> None
   :noindex:
   :module: arbor

   A location on a cable cell.

   A location on :attr:`branch`, where :attr:`pos`, in the range ``0 ≤ pos ≤ 1``,
   gives the relative position
   between the proximal and distal ends of the branch. The position is in terms
   of branch path length, so for example, on a branch of path length 100 μm ``pos=0.2``
   corresponds to 20 μm and 80 μm from the proximal and distal ends of the
   branch respectively.


   .. py:method:: location.branch() -> int
      :noindex:
      :module: arbor
      :property:

      The id of the branch.


   .. py:method:: location.pos() -> float
      :noindex:
      :module: arbor
      :property:

      The relative position on the branch (∈ [0.,1.], where 0. means proximal and 1. distal).

Which produces acceptable output:

Updating is manual, but by checking the full reference in, a difftool can be used.

[thorstenhater]

Hi,

this has been dormant for quite some time, what's the current stance on this?

[brenthuisman]

Took a tour of the above linked blocking issues/PRs, no progress:

python/cpython#83306
pybind/pybind11#2621
sphinx-doc/sphinx#4824