PTB goes asyncio in v20!

[Bibo-Joshi]

We are embracing the future: asyncio support is on its way and it's gonna be legen, wait for it, -dary!

For those of you who don't know what asyncio does: When calling e.g. bot.send_message(…), your code spends a lot of time waiting for the Telegram API to respond. During that waiting time, other code could be running. So far this was achieved in PTB using multi threading. But:

By now asyncio is a well established standard for concurrency in networking libraries. It can be faster than threaded code and provides some more flexibility. Until the beginning of 2020, PTB still supported Python 2.7, but by now a year has passed and we see an increasing amount of questions on how to integrate asyncio code with PTB. On top of that, we have some plans for features that would heavily benefit from a switch to asyncio.

That is why with the release of version 20, python-telegram-bot will go asyncio!

We are aware that this will be a major breaking change, as

• all callback definitions will have to change from def callback … to async def callback,
• a lot of calls of bot methods will have to change (e.g. message = bot.send_message(…) to message = await bot.send_message(…)) and
• projects that use python-telegram-bot in a multi-threaded manner will have to be reworked

However, this switch will also mean a lot of work on the internals of our library and trying to make the change backwards compatible would mean a huge additional effort. We will therefore not provide a compatibility layer. Making this decision was not easy for us, but we hope that it enables us to put our energy in further improvements and enhancements of python-telegram-bot rather than maintaining legacy code for backward of compatibility.

This is why we're making this announcement that early in our development process: We don't know when we will release version 20. It may be in a few months or in 2 years - as you know we don't like to give ETAs. But it's important to us to involve the community in the transition and give you a chance to prepare. We will make another announcement when we have an asyncio branch ready. We will give you the opportunity to try the new PTB out and will try to support community based compatibility tools. The actual release will happen later and will be announced beforehand as well.

We hope that this switch is in your interest. If you have questions or feedback about the announcement, feel free to comment on this GitHub thread. We will try our best to answer your questions and take your comments into account.

While we prepared the switch to asyncio, we took the chance and did some much needed spring cleaning in the library. While this means a more consistent user appearance, it also means a lot of breaking changes. We will write a transition guide, as always.

In the meantime you can expect some non-breaking improvements. Should Telegram release another API update, we will update our library before going ahead with version 20.

As of now, all development to the main branch has been stopped. If an API update happens or some mayor bug appears, we will release another version, otherwise the next proper release will be V20.

Your PTB Developer Team

»Dread it. Run from it. asyncio still arrives.« - Thanos, ca. 2018, allegedly

[arun-mani-j]

When PTB 14 is released, we should change our code to async style or it will optional to switch ?

For example Rust's Reqwests library has support for both 🤔

[kopyl]

Second this. Will PTB be async-only and if I don't know async I won't be able to write bots on PTB?

If it will be async-only, why should one choose PTB over aiogram?

[Poolitzer]

change our code to async style or it will optional to switch

you dont have to change "your" code to async, all calls for this library have to be though. That is what the seventh paragraph talks about.

If it will be async-only, why should one choose PTB over aiogram?

we provide a very different interface, a bigger documentation (as far as I can see), well integrated jobs. But we won't stop you from using aiogram ofc, do what makes you happy :D

[Bibo-Joshi]

if a callback doesn't make any asyncio calls and can be defined as non-async, I guess it's reasonable to try and support that. I.e. do something like

if inspect.iscoroutinefunction(callback):
    return await callback(…)
return callback(…)

in Handler.handle_update

[91DarioDev]

TL;DR please consider a way to process handlers of an update in different groups synchronously like now (without run_async), but process updates asynchronously between other updates.

Please consider to add a way to process the group handlers synchronously during an update (like if they were a single function and like they now work without run_async), while handling updates asynchronously between them.

this because a developer could not mind that updates are processed one by one, but he may be interested that group handlers are processed synchronously because you may have a group that add the users to the database processing some data, and in the second group you may send a message to that user fetching the data from database added by the previous handler.

Basically processing group handlers synchronously it’s like now it works without adding the run_async params to handlers.
But to be faster you can process updates contemporary if you don’t have problems of race conditions or if -like in my case- race conditions are already handled by the code.

Thank you for the great job you do ❤️

[Bibo-Joshi]

Hi, thanks for the comment :) We haven't discussed the details of how tg.ext will change yet, but IMO that's pretty much what should happen, i.e. the run_async parameter should pretty much do the same in v14. See also the third § here #2288 (comment) :)

[Khodunov]

Hi
I heavily used your library because at first I was at beginner level and your library is perfect for beginners: for instance, there is no need to understand asyncio to start creating bots.
Now I use your library when teaching students, and for that matter it was perfect: it didn’t use decorators or asyncio, had simple logic and thus could be used by a beginner to create a simple bot (and get highly motivated to continue learning python). It seems that after the 20.x release, the most used and supported telegram-bot libraries are becoming non beginner-friendly :(
Don’t know whether you thought about this aspect, but for me it matters greatly, so could you please at least think about adding a synchronous mode for you library?

[Poolitzer]

Why is asynchronous programming not beginner friendly? If you dont want to explain why you have to write async before functions and call methods with await, don't? Thats literally the only change one has to make in non complex/beginner code. Apart from updater/dispatcher now being even easier to understand because there is one application which holds everything together.

[harshil21]

Hi, thanks for the concern. We decided not to offer synchronous support since we found out that simply adding an async/await to your callbacks does not make it harder for beginners (among other reasons). One doesn't need to have a deep understanding of asyncio to start writing bots or learning python. You could tell the students that await calls async functions for now.

The switch to asyncio is very advantageous for making bots (see OP) which is the main selling point here.

Besides, as poolitzer mentioned, there are many other parts of the code which were simplified and have become easier to understand too.
However that being said, we are still offering support for v13.x versions (API updates and critical bugs) so you can also continue using that for teaching.

[Khodunov]

Why is asynchronous programming not beginner friendly?

Well, because it’s a new concept to grasp — and not the basic one (you could easily never touch asyncio if you’re not a professional)

If you dont want to explain why you have to write async before functions and call methods with await, don't

Yeah, that’s how to teach programming without teaching to understand :(
When you just start writing code in Python, everything seems very simple and it motivates to continue. You understand everything that’s happening in your code, each symbol having its own meaning — and when you start seeing the module you’ve never seen before (and it adds strange words to all the functions somehow and the logic is not linear anymore), it makes you not want to learn that anymore. For instance, Django is bad for beginners (at least in my experience), but Flask and PTB v13.x are great.

However that being said, we are still offering support for v13.x versions (API updates and critical bugs) so you can also continue using that for teaching

I’ll probably go that way for the most of the students, but will leave v20.x and asyncio as the second level there (because at the simple level you don’t really need to think deeply about what’s happening there)

Thank you for your replies :)