Canonical way to add builtins docstring support?

[mikez]

I'm using pyright directly, not via pylance.

What is the canonical way to have the language server return docstrings for builtins?

One hacky thing I tried was adding a custom builtins.py to extraPaths with some example docstrings and this seemed to work. (Adding a custom stub builtins.pyi did not.) One might hereby generate the docstrings via .__doc__ or pydoc.getdoc; or, arguably even better, via the official documentation at https://docs.python.org/3/library/stdtypes.html if viable.

It seems to me, there must be an easier way though.

Thank you @msfterictraut and team for your wonderful work with pyright. Much appreciate it.

[jfcherng]

A bad news for you #4250 (comment)

[mikez]

@jfcherng Thanks for the pointer. I now more clearly understand that Pyright is most concerned with type checking functionality. This separation of concerns makes sense to me.

However, my question remains—is the "hacky" way I outlined the most canonical way to add custom builtin-docstrings if one can't use Pylance? It seems Pylance uses Pyright as its "core typing engine" and thus also uses its API?

[erictraut]

If you want this capability, the recommendation is to switch to pylance, which adds this and many other language server features on top of pyright.

There is no "canonical way" to add features to pyright that it was not designed to support. It sounds like you've found a way to get the behavior you're looking for.

[jfcherng]

This reminds me that there is https://github.com/DetachHead/basedpyright, which may be interested in some other non-targeted feature for pyright.

[mikez]

@erictraut Thank you for the fast response amidst my ignorance.

I think I see the bigger picture now. Docstrings were a by-product of the typechecker, never the aim. Thus, they are a "sometimes"-working feature of pyright, but not a supported feature. Eventually, one might decouple the docstring-related code to a separate repository and take it out of pyright all-together, but for now it's still in there as a bonus. Also, remarks such as in https://github.com/microsoft/pyright/blob/main/docs/typed-libraries.md#docstrings where it says "It is recommended that docstrings be provided for all classes, functions, and methods in the interface." — this is meant in particular in light of usage with pylance and the like with docstring support per design... the comment wasn't meant for pyright. Also, you don't follow this advice in pyright-bundled stubs because docstrings are not the chief aim and out of scope.

[erictraut]

Pyright extracts docstrings from stub files and source code when they're available. However, the stubs that describe the interface to the stdlib come from the community-maintained typeshed project, and these stubs don't contain any docstrings. If the typeshed stubs were to include docstrings, pyright would take advantage of them.

Pylance mitigates this by bundling separate stdlib docstrings that have been extracted via runtime introspection, and it adds logic that uses these bundled docstrings where appropriate in the language server.

[mikez]

Thanks for the pointers, @erictraut. Now I see the catch-22 of the situation. This thread was insightful, esp. @gramster's pointer to docs2stubs and Guido's point of the maintenance burden if it were in typeshed.

I also noticed that the mypy project has a tool stubgen --include-docstrings …. It doesn't seem like a perfect drop-in replacement to put in "typeshedPath", but maybe it can get there.