index for doxygengroup

[rscohn2]

I would like the contents of my doxygengroup to be in the table of contents so I could do something like this by making doxygen groups for each of the major categories. The link is not my document, but I have something similar.

You hint that it might be possible here

If the code changes are not too complicated and you could sketch out what is needed, I could try.

[vermeeren]

Although breathe cannot do this directly iirc, I believe either breathe-apidoc or exhale should be able to generate the TOC automatically in a way similar to the example posted.

Could you check if the info in #293 helps you further?

[rscohn2]

I stopped using exhale because it seemed to be blowing up the processing time from 20 seconds to 30 minutes. I am on windows and suspect the problem is related to creating many small files. doxygengroup puts the whole group in 1 file, which made it easy to control the number of files. Even without the slowdown, my preference is to put groups in the same web page.

I need to try exhale again to verify that it was the cause of the big slowdown.

It does look like #293 will help if I am able to put each item in a separate file.

[rscohn2]

I timed the options:

• doxygenindex: 30 seconds
• breathe-apidoc: 160 seconds
• exhale: 1060 seconds

doxygenindex/doxygengroup/etc:

• flexible about including API lists in a doc with other text surrounding it
• can put multiple API in a single web page
• Is much faster

The only drawback (for me) is that it does not include the contents in the TOC. I want to do something like this:

.. doxygengroup:: foobar
   :heading: -

And it would emit the group, creating a heading

funcname
--------
.. doxygenfunction:: funcname

It might be possible to post-process the RST output of breathe-apidoc into the formatting I want, but it seems like a hack and better to have the functionality directly implemented. Can you point me at something similar in the code that would help me to understand what is needed?

[vermeeren]

Some small remark: Sphinx can build in parallel (if the extensions used support this too) with -j auto. Although likely still not as fast as doxygenindex, it should help a lot with apidoc and exhale where many source files are generated. Have you used this option yet?

The apidoc script is mostly standalone afaik, see breathe/apidoc.py.

[rscohn2]

I did not know about -j auto and have added it to my scripts.

I looked at breathe/apidoc.py. My understanding is that it generates RST files which uses breathe and sphinx directives. I would like to instead change the behavior of doxygengroup/doxygenfile so they can optionally wrap the elements in sections so they will appear in a TOC. It will be faster and more flexible in the structure of the document.

I think the change is simple for someone who understands the internal representations of markup used by sphinx & breathe, but I am having trouble understanding how it works. I see how sphinxrenderer.py transforms the internal representations. It would help me if I could write some examples and have a debugging option dump out text versions of the internal representation of all the nodes. Do you know how to do that?

[vermeeren]

Your understanding of apidoc is correct. The changes proposed to doxygengroup/doxygenfile sound good. Unfortunately I don't know much about Sphinx's internals or how debugging works with it. It appears docutils has something that may be useful at https://docutils.readthedocs.io/en/sphinx-docs/user/tools.html#testing-debugging-tools (rst2pseudoxml.py).

@jakobandersen may know of some better ways to do development.

[jakobandersen]

Note that there is sphinx-doc/sphinx#6316. It may be that a resolution for that issue would resolve yours? (though I don't think it is close to be resolved)
Some semi-random points:

• Actually, Create a ToC entry for every function, method, class, etc sphinx-doc/sphinx#6316 calls for only adding a TOC entry, but you suggest a ordinary sectioning construct.
• What should the title of each section be? Specifically in the case of function overloads. Perhaps we can look at what text is used on the index page for inspiration, and perhaps that should be improved as well.
• To implement your suggestion, based on https://docutils.sourceforge.io/docs/ref/doctree.html#section it looks like one in principle "just" have to wrap the node at https://github.com/sphinx-doc/sphinx/blob/8e1cbd24c61934df7eb426aad0dc48830789b096/sphinx/directives/__init__.py#L157 with a sectioning node.
• Not all declarations are eligible for 'sectionization', In which nested directives can Sphinx C++ directives and Breathe directives appear besides them self? To detect the proper cases we may need to look at the state machine stuff in Docutils.

[rscohn2]

Note that there is sphinx-doc/sphinx#6316. It may be that a resolution for that issue would resolve yours? (though I don't think it is close to be resolved)

It's close. I could not turn the option on globally, so I would need a mechanism to turn it on for a set of definitions. Breathe directives like doxygengroup seemed like it would work, but other ways would work.

What should the title of each section be? Specifically in the case of function overloads. Perhaps we can look at what text is used on the index page for inspiration, and perhaps that should be improved as well.

Yes, using the index page text would be reasonable.

Not all declarations are eligible for 'sectionization', In which nested directives can Sphinx C++ directives and Breathe directives appear besides them self? To detect the proper cases we may need to look at the state machine stuff in Docutils.

I am not sure what to do with nested directives.

To implement your suggestion, based on https://docutils.sourceforge.io/docs/ref/doctree.html#section it looks like one in principle "just" have to wrap the node at https://github.com/sphinx-doc/sphinx/blob/8e1cbd24c61934df7eb426aad0dc48830789b096/sphinx/directives/__init__.py#L157 with a sectioning node.

Should I continue to look into adding it to breathe, or do you prefer that it goes into sphinx directly?