register custom autosummary get_documenter functions

[keewis]

Feature or Bugfix

• Feature

Purpose

• refs: closes Autosummary with custom documenters #7908
• allow registering custom get_documenter functions

This is a rough draft of the feature I proposed in #7908. Using this to override get_documenter would allow me to rewrite my hack to something like

def get_documenter_from_template(autosummary, obj, parent, real_name):
    ...

and then to register it:

def setup(app):
    app.add_autosummary_get_documenter(get_documenter_from_template)

I don't really need all the information from the Autosummary class (only autosummary.env.app, autosummary.config.autosummary_context (maybe? I don't really understand that value), autosummary.options and real_name), but I thought the hook should be pretty general so it doesn't only solve my own use case. So as long as I have access to that information, I'm happy to change the signature.

Also, if there's any way to solve my problem without overriding get_documenter (I can't use can_document_member and priority because there's no way to decide from obj whether the documenter is appropriate), I won't insist on this feature.

[keewis]

@tk0miya: any chance this can be included in 3.2.0? It would really help me with fixing my extension, but I totally understand if you want to think about this for a bit longer.

[tk0miya]

I can't imagine what extension this helps except yours. I understand your extension need to access the template option of autosummary directive. But I think they should not be related. So I can't allow merging this change. Only I can do is making get_documenter replaceable easily in subclasses for overriding the directive by extensions:

class Autosummary:
    def get_items(...):
        ...
        doccls = self.get_documenter(self.env.app, obj, parent)
        ...

    def get_documenter(self, app, obj, parent):
        return get_documenter(app, obj, parent)  # Extensions can override this method and overide the directive forcedly.

[keewis]

that would work, too, but it would make all extensions using this mechanism conflict with each other. Not sure if that's an issue, though.

[keewis]

I just noticed that moving get_documenter to Autosummary is not possible because some functions in sphinx.ext.autosummary.generate use it and don't have access to a Autosummary object. How about adding a wrapping method? That way my proposed changes would become just

diff --git a/sphinx/ext/autosummary/__init__.py b/sphinx/ext/autosummary/__init__.py
index c1d0febe0..32d49ca29 100644
--- a/sphinx/ext/autosummary/__init__.py
+++ b/sphinx/ext/autosummary/__init__.py
@@ -281,6 +281,10 @@ class Autosummary(SphinxDirective):
 
         return nodes
 
+    def get_documenter(self, app: Sphinx, obj: Any,
+                       parent: Any, name: str) -> "Type[Documenter]":
+        return get_documenter(app, obj, parent)
+
     def get_items(self, names: List[str]) -> List[Tuple[str, str, str, str]]:
         """Try to import the given names, and return a list of
         ``[(name, signature, summary_string, real_name), ...]``.
@@ -313,7 +317,7 @@ class Autosummary(SphinxDirective):
                 full_name = modname + '::' + full_name[len(modname) + 1:]
             # NB. using full_name here is important, since Documenters
             #     handle module prefixes slightly differently
-            doccls = get_documenter(self.env.app, obj, parent)
+            doccls = self.get_documenter(self.env.app, obj, parent, real_name)
             documenter = doccls(self.bridge, full_name)
             if not documenter.parse_name():
                 logger.warning(__('failed to parse name %s'), real_name,

(I seem to really need real_name for filling in the template)

[tk0miya]

Sorry, I can't understand your proposal. Even if we added real_name to the parameters, sphinx.ext.autosummary.generate is not affected because they are not related to the Autosummary directive object.

The generate_autosummary_* functions surely scan the reST parses and detect autosummary directive calls. But they don't use reST parsers. So no directive object is instantiated on parsing. So the change of Autosummary directive class does not affect the functions at all.

[tk0miya]

I feel strange to add real_name to the get_documenter() method. But I can accept if we change it to create_documenter() method. It's understandable.

[keewis]

some functions in sphinx.ext.autosummary.generate call sphinx.ext.autosummary.get_documenter (the function), so if we remove that / convert it to a method on Autosummary, those functions are going to break.

But I can accept if we change it to create_documenter() method

sure, that sounds good.

[tk0miya]

Understand. I didn't mean to remove sphinx.ext.autosummary:get_documenter() function itself. I only proposed you to add a new method Autosummary.get_documenter() as a helper to override it for you (at that time).

[keewis]

okay, then that was my bad. I'll rewrite the PR.

[keewis]

If the way it is now is acceptable, I think we should discuss the way this is meant to be overridden. While forcibly overriding the Autosummary class used by the autosummary directive is certainly possible, it means that only a single extension using this mechanism can be used. Instead, we could provide a function to register it:

def register_create_documenter(func):
    original_func = Autosummary.create_documenter

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        if result is not NotImplemented:
            return result
        else:
            return original_func(*args, **kwargs)

    Autosummary.create_documenter = wrapper
    return wrapper

of course, that will still break as soon as someone overrides the autosummary directive using a custom class that does not inherit from Autosummary or Autosummary is converted to use __slots__.

Is this something you would consider including?

[tk0miya]

Is this something you would consider including?

No. It's too much. Let's discuss again after another extension appeared. I can't imagine the extension what needs to hook this. I just made a new method only for you.

[keewis]

Let's discuss again after another extension appeared

sure

I just made a new method only for you

thank you very much for that!

[tk0miya]

Merged!

[keewis]

thanks a lot, @tk0miya. 🎉