Make documentation more accessible

[Bibo-Joshi]

Hello world.

We receive regularly queries in our help group from people unable to read the support documents we provide. We believe this is mainly because we don't do a good enough job presenting them. This is where you come in: If you are new to the project and think "hey, this can be improved", and its not on the list yet, tell us about this.

One maintainer has compromised the thoughts we as maintainers made about this topic, but we value foreign input very much. Some discussion about this happened at https://t.me/pythontelegrambotgroup/504413 + answers.

edit: The list was extended with the ideas from the comments below and converted into a checklist so that it's easier to keep track of what's already been done.

The mentioned ideas:

• Cross referencing. a LOT: -> [Documentation] Add more cross-references #3110

  • within RTD: JobQueue -> CallbackContext.job_queue, Dispatcher.job_queue, Updater.job_queue. Bot.edit_message_text -> Message.edit_text. Similar for all other stuff that makes sense - bascically all "where is this used" references. sphinx has the nice .. seealso:: directive that we can use for this Docs cross references (#2633) #2855
  • examples + RTD -> wiki: JobQueue -> wiki/jobqueue, Bot.arbitrary_callback_data -> wiki/arbitrary_callback_data. examples/timerbot -> jobqueue. etc.
  • RT + wiki -> examples. JobQueue -> timerbot.py, wiki/jobqueue -> timerbot.py etc.

  These should help a great deal when it comes to "where do I use this"? or "how is this used"?

• Maybe: On RTD, for the really newbies a "how to read sphinx docs", e.g. explain that "Parameters: token (str) – Bot’s unique authentication." means that the keyword argument ist named "token" and expects input of type str

• We have several notes and docstrings that are repeated very often across the docs, which makes it hard to maintain them. An idea is to use the .. include:: directive to unify those. Unfortunately this currently doesn't work with the napoleon docstring style that we use - see Resolve directives before running napoleon sphinx-doc/sphinx#9939. If/when this can be tackled, this should include the following:

  • Unify documentation on how to pass files. Some of the classes/methods that accept file input have a note like here, but others don't. Also that note is outdated, as it doesn't explain when a pathlib.Path can be passed. It should also mention that we don't support open(file, 'r') but only open(file, 'rb'). We should either have a unified note that we add everywhere or move this snippets section to a standalone page, extend it a bit and point to it from the docs. -> [Feature] Improve handling of local mode & docs regarding file handling #3114
  • Add a note to all Bot.edit_* methods that states the limitation about editing messages from the official docs: -> [Documentation] Add note on editing with reply markups #3111

    Please note, that it is currently only possible to edit messages without reply_markup or with inline keyboards.

• Improve docs of ConversationHandler regarding the TIMEOUT & WAITING state, probably including a small code example. This is one of the things that in my experience users have the hardest time to grasp. -> let's do this in the context of [Discussion:] ConversationHandler #2770

• Add valuable information to attribute docstrings of telegram classes Make documentation more accessible #2633 (comment) -> [Documentation] Add valuable information to attribute docstrings of telegram classes #3109

• Think about making use of blacken-docs Edit: Currently is not easy to integrate into workflow. not overly important anyway.

• List changelog in the resources in the readme Documentation Improvements #3103

• Make documentation about the "bots default timezone" more explicit - i.e. explain better that naive (date)time objects will usually be interpreted as UTC Documentation Improvements #3103

• Update docstrings of attributes which are a list, but may be empty. See Update _message.py - issue #2606 #2741 (comment) Empty lists #3057

• Find a way to document Generic classes better, i.e. document which types they need to be fully annotated

• On the start page of RTD, the wiki, the examples folder & the GH readme, explain better what the different resources cover. RTD covers technical reference, i.e. comprehensive but not too elaborate, wiki explains certain features in detail, examples do minimal code showcases with little explanation. I imagine a somewhat unified text like "You're at the examples page, which gives you minimal code showcases for PTB features. For detailed tutorials on certain PTB features, please head to the wiki. …" and similar for the other resources.

• In the docstrings of Update.effective_*, explicitly explain what those attributes will be set to. See [FEATURE] A function to automatically decide if update.message.reply_text or update.callback_query.edit_message_text will be used. #2826 (comment)

• Link classes to python to docs. in the above mentioned thread, the user was not aware how to build a datetime object and tbh one can't expect newbies to know about all python std-libs. in sphinx it's rather easy to cross-ref to other sphinx docs, so let's do that. Docs link python classes to docs #2633 #2739

• get rid of all the JSON-serialized that we copied from the offiicial docs - PTB users never encounter JSON-serialized objects. I think we briefly discussed this at some point but never acted on it …

• Document that TelegramObject is subsriptable

• Ideas for catching errors in documentation faster and easier: Docs improvements: referencing #2798

  • Enable the nitpicky option. We can do this after we enable intersphinx since there are lots of 'warnings' now Docs improvements: referencing #2798
  • add a workflow that builds the docs and checks for warnings & errors Docs improvements: referencing #2798

• Make parameters referencable with the sphinx-paramlinks extension Docs improvements: referencing #2798

Documentation is a great place to get started contributing to Open Source projects without having to write/dive deep into the complex code making up this project. If you look for an "easy" way to give back to the OS community this October, this might be it!

If you like to start working on one of the ideas, comment on this issue as well and say what you want to work on. Once you are ready to show your changes, open a PR towards the doc-fixes branch.

[Bibo-Joshi]

4 more ideas:

1. get rid of all the JSON-serialized that we copied from the offiicial docs - PTB users never encounter JSON-serialized objects. I think we briefly discussed this at some point but never acted on it …
2. Unify documentation on how to pass files. Some of the classes/methods that accept file input have a note like here, but others don't. Also that note is outdated, as it doesn't explain when a pathlib.Path can be passed. We should either have a unified note that we add everywhere or move this snippets section to a standalone page, extend it a bit and point to it from the docs. I know that I voted against this particular linking before (https://t.me/c/1101839433/7824), but I've changed my stand regarding linking to the wiki from the docs.
3. Document that TelegramObject is subsriptable
4. Improve docs of ConversationHandler regarding the TIMEOUT & WAITING state, probably including a small code example. This is one of the things that in my experience users have the hardest time to grasp.

[Poolitzer]

I will manage this issue through Hacktober. If you feel the need to discuss something about this in private for some reason, feel free to PM me. Otherwise happy hacking!

Regarding this issue, I also thought that we have too many repeating notes we have to change everywhere (and potentially forget doing that in some placed if we have to). I think the fitting directive would be include https://docutils.sourceforge.io/docs/ref/rst/directives.html#include. Using this, we should be able to write the note(s) in one/several place/s and include it everywhere we want. That way its generated everywhere it currently gets generated, but its only written in one place, so we dont forget changing it anywhere.

[Bibo-Joshi]

Another idea (inspired by #2678), that would also be very well suited for the inlude directive: Add a note to all Bot.edit_* methods that states the limitation about editing messages from the official docs:

Please note, that it is currently only possible to edit messages without reply_markup or with inline keyboards.

[Bibo-Joshi]

I openend a branch called doc-fixes. PRs for this issue should be based against that branch so that we can gather all contributions in one branch before merging into v14. See also #2693

[kaustubhgupta]

Hello,

I am using this package for my projects and I like the wrapper implementation. I want to contribute towards:

• Creating a tutorial/examples section. The main aim of this section would be to explain some of the basic bots that can be built using this package. It would center around how to quickly add handlers to handle user input and build upon a simple bot to complex ones. I know the wiki page has this information but I think it should be shifted or cross-referencing to the docs too.

• Creating a articles/resources section. Here, all the articles, blogs, guides that are made to use this package can be listed. This section can be helpful for people who are looking for articles where authors explain the in and out of their bots. Plus listing authors' work will support them and in turn, new authors will get a push to publish new articles to be featured in the docs.

Let me know your thoughts on these ideas.

[Poolitzer]

Hi there.

Thank you very much for your intention to contribute! I am not sure I understand your points though. The tutorials do live in the wiki page and the examples are in the examples folder. What do you want to change specifically to this?

Towards the 3rd party resource section, I am also not sure where you imagine this should go. Nonetheless, there is a wiki page kinda doing that: Related projects. I am kind of hesitant with putting too much effort into extending this section though. 3rd party code tutorials tend to get outdated quickly, which is totally understandable since the author has moved on, but it hurts the project a bit. People following these tutorials run into problems because we broke some way the code relied on, and then they come to the support group and ask us why its broken. We dont have this issue with our tutorials since we update them all the time, so that is where I would put the focus.

[harshil21]

Another idea to catch errors in documentation faster and easier: Enable the nitpicky option. We can do this after we enable intersphinx since there are lots of 'warnings' now

[Bibo-Joshi]

We could even add a workflow that builds the docs and checks for warnings & errors 🤔 Probably only worth the effort after #2178 …

[Poolitzer]

good idea.

[eldbud]

2. Link classes to python to docs. in the above mentioned thread, the user was not aware how to build a datetime object and tbh one can't expect newbies to know about all python std-libs. in sphinx it's rather easy to cross-ref to other sphinx docs, so let's do that.

i just opened a pull request - #2738 - which will resolve this item

[Bibo-Joshi]

eldbud running sphinx with nitpicky in #2738 made me aware that we use something like :attr:`argname` in a lot of places to reference arguments of functions because it renders more nicely than ``argname``. However, this leads to a lot of unresolved references which nitpicky complains about.

A straightforward solution would be to switch to ``argname``. Another solution would be to use sphinx-paramlinks package, which makes parameters part of the index. I tried it a little bit and it seems to work fine even with sphinx.ext.napoleon which we use. So far I only found one bug, but that's easy to patch (see sqlalchemyorg/sphinx-paramlinks#11).

Please for now understand this as note, not as something that I'm determined to do. For example, it would be worth to first check if using the extension allows to reference parameters from roolsbot - this is something that I would finde extremely useful, but sqlalchemyorg/sphinx-paramlinks#7 could indicate that this is not directly doable …

[Bibo-Joshi]

Short update regarding sphinx-paramlinks: rools works fine with that. We'd just need to make sure that one can distinguish e.g. ConversationHandler.states from ConversationHandler.params.states in the search, but that's a minor problem I guess. Also I send sqlalchemyorg/sphinx-paramlinks#12 to close sqlalchemyorg/sphinx-paramlinks#11 and the author seems to be willing to merge it once they could test it :)

[Bibo-Joshi]

About the .. include:: thing: Apparently that's not as easy as we thought. See sphinx-doc/sphinx#9939

[Bibo-Joshi]

Another item: The docs of Telegram classes are currently written such that the docstring of the arguments are more informative than the docstring of the respective attributes. E.g. for Chat.has_private_attribute, "Returned only in get_chat()" is only present in the argument docstrings, not in the attribute docstring. For Messag.text, the length limitation is only documented for the argument, not the attribute.

This logic makes sense for classes that are to be instantiated by the user (like BotCommand or most everything within tg.ext), but for classes that are rarely instantiated by the user like Chat and Message this makes it sometimes hard to find valuable information.

I suggest to revisit the documentation of the Telegram classes and either unify the docstrings of args & attributes or at least check which information would be worth adding to the attributes as well.

I'll be adding this to above todo list as well, but wanted to elaborate here.

[Bibo-Joshi]

Almost all items of this issue have been addressed or converted into standalone issues. closing.