Multiple MessageHandlers conflict in the same ConversationHandler

[tomasjur]

Hi, I will imagine and create a demo situation by replacing my existing names to explain the issue I am currently facing and not finding a solution for it. I will leave a concept similar.

Problem

My bot needs to always (almost) check whether the user pasted a car name that is being validated outside (in another file). It can be pasted anytime randomly and I want it to handle it. It's one of the entry points, so the user can jump to the Conversation Handler anytime by pasting the car name.

It fails when:

1. I go with /start, and from the buttons choose to add a new licence plate (PLATE).
2. ask_for_licence_plate is being called because START_OPTIONS have this set.
3. I enter licence plate and submit.
4. It ends up in check function instead of add_plate. This is because the entry point and ADD_CAR_LICENCE_PLATE have the same MessageHandler even though the correct state is being returned.

Code

<...>

async def check(update: Update, context: ContextTypes.DEFAULT_TYPE) -> int:
    if await api.is_car_name(update.effective_message.text):
        context.user_data["car_name"] = update.effective_message.text
        return await get_info(update, context)
    else:
        await context.bot.send_message(chat_id=update.effective_chat.id, text="If you want to get info, enter a car name!")
        return ConversationHandler.END
 
async def ask_for_licence_plate(update: Update, context: ContextTypes.DEFAULT_TYPE):
    await context.bot.send_message(chat_id=update.effective_chat.id, text="Enter the car licence plate:", reply_markup=ForceReply())
    return ADD_CAR_LICENCE_PLATE

<...>

conv_handler = ConversationHandler(
        entry_points = [CommandHandler('start', start),
                        MessageHandler(filters.TEXT & ~filters.COMMAND, check),
                        ],
        states={
            START_OPTIONS: [
                CallbackQueryHandler(ask_for_licence_plate, pattern="^" + str(PLATE) + "$"),
                CallbackQueryHandler(history, pattern="^" + str(HISTORY) + "$"),
                CallbackQueryHandler(settings, pattern="^" + str(SETTINGS) + "$"),
                CallbackQueryHandler(tools, pattern="^" + str(TOOLS) + "$"),
                CallbackQueryHandler(help_command, pattern="^" + str(HELP) + "$"),
                CallbackQueryHandler(close, pattern="^" + str(CLOSE) + "$"),
                CallbackQueryHandler(start, pattern="^" + str(START) + "$"),
            ],
            ADD_CAR_LICENCE_PLATE: [MessageHandler(filters.TEXT & ~filters.COMMAND, add_plate)],
            HELP_OPTIONS: [
                CallbackQueryHandler(start, pattern="^" + str(START) + "$"),
                CallbackQueryHandler(close, pattern="^" + str(CLOSE) + "$"),
            ],
        },
        fallbacks = [CommandHandler('close', close)],
        allow_reentry = True
    )
    application.add_handler(conv_handler)

What have I tried?

1. Disable allow_reentry. This helps to not use entry points anymore, but it doesn't work when randomly pasting the car name. And also if I would like to have /help command and add it to the entry points list, it won't work because it doesn't get back to entry points anymore. So I would need to repeat CommandHandler multiple times, that's not efficient.
2. Use application.add_handler separately outside ConversationHandler. This looks promising, however, when I jump from check to get_info and then would return anything like START_OPTIONS, it shows no errors and not do anything at all. I assume that's because it's not in the conversation flow.
3. Use Regex. Okay, I have even more MessageHandlers, so it helps for some, but if I ask for user car name to save it to the database, it still has the same Regex as an entry point.

And some other methods that I cannot remember now.
But this driving me nuts, so I decided to write here.
I know it might look confusing, but the real problem is that it confuses which MessageHandler to call. I'd like to always check if the written message is the car name, but not only when I actually ask the user for text to interact with.

[tomasjur]

By the way, my real-life case is that I want to allow users either to go through the full path by clicking buttons: Start -> Car -> Get Info By Car Name -> Provide Car Name -> Get Info From Bot or just paste the car name and get information about it immediately if user wants. This makes user experience better.

[Bibo-Joshi]

With allow_entry=True, the entry points are currently checked before the current states handlers are checked. This is already tracked as improvement topic in #2770.

[tomasjur]

Any ETA on this?
Also what is the workaround for now?

[Bibo-Joshi]

No, there is no ETA.
The only current workaround I can think of (apart from patching/implementing your own ConversationHandler) is to endure that the entry points can not catch updates that are valid in the states

[tomasjur]

Until there is a fix for this, I temporarily edited the conversationhandler.py code on lines 780 to 809.

1. Switched if (line 781) with another if (line 793) places to first check for states, not entry points.
2. Changed if state is None or self.allow_reentry: (line 781) to if (state is None or self.allow_reentry) and handler is None:.
3. Commented out (or completely remove) else: return None (line 808) because we want to move to entry points if we don't find the stage or fallback needed.

Whoever will face the same issue and read this, use this workaround at your own risk. Also, there is a smarter way by rewriting this part of the code but I already spent more than needed time on this particular small problem.

As far as I tested now with commands, randomly pasting messages, and replying to the bot when asked, everything works as expected because it first checks for states and then for entry points, not vice versa as it is right now.