I can set a retry timeout for failed tasks

[saig0]

AT:

• when a task is marked as failed, I can set a timespan after this the task can be locked again
• if no time is set then the task can be locked immediately

[kunalkrishna1111]

We need a dynamic update to the timer. Please prioritize.

[sergeylebed]

When we can expect this feature to be implemented?

[Zelldon]

@npepinpe

[npepinpe]

This issue was opened before I'd even heard of Zeebe (:sweat_smile:), so let me recap to make sure I understand the intent:

Once a task exhausts its retries and is marked as failed (so not activatable/lockable anymore), we want to set a timeout such that the related incident is automatically resolved by increasing the number of retries?

I imagine the use case is for some sort of back off, e.g. try 3 times, wait 5 seconds, try again 3 times, without the need for any extra command/action in between?

[Zelldon]

I think it is not about incident resolving. What I understood was that it is about to make it available again after an X timeout. Currently it is immediately available again for activation after one worker failed it and there are retries left, but this is problematic if the reason is an external system which might be down for a while. This will cause then an incident, since it will directly fail again. If we had back off retries we would avoid some of these cases.

@saig0 correct me if I'm wrong

[npepinpe]

Ah, so it's not when the task is marked as failed (e.g. its state is failed) but just when we get a fail command?

[npepinpe]

Both @Zelldon and @saig0 can help you out there (if one is too busy)

[aivinog1]

Hi @Zelldon @saig0! Can you give me some insights into this? :)

As I understand needs to make a task available after some backoff policy configured to it, am I right? :)

And maybe you know some pitfalls or helpers for me? :)

[saig0]

Yes, correct. The idea is that a client (Java/Go/Zbctl) can specify an optional backoff time when sending a JobFailed command. If a backoff time is set then the job will be available after the backoff time. Otherwise, the job will be available immediately (current behavior).

The internal logic in the broker can be implemented in a similar way as DueDateTimerChecker.

The fail command is handled by the FailProcessor.

You can see how a job a marked as activated again, for example, in the TimeOutProcessor.

Please ask questions if the requirement or the code is unclear. Thank you 🍪

Since this is a bigger change, I would suggest splitting the broker part and the different client parts into separate PRs.

[aivinog1]

@saig0 Thanks, I got it :) I think there will be three PRs for that issue.

[npepinpe]

I'm downgrading this for now as we're refactoring our engine and it doesn't make sense to do this at the moment.

[aivinog1]

Hey @npepinpe and @saig0! Is the engine's refactoring ended? May I return to this task? :)

[saig0]

@aivinog1 yes, it's done 🎉 Feel free to work on it 🚀

[saig0]

The feature is available but not exposed to the clients yet. We should implement #5629 to complete the feature.

[aivinog1]

@saig0 I guess we can close this since #5629 closed and it was implemented in the Java and Go clients.

[saig0]

@aivinog1 yes 🎉 Good job 🎖️

I'll create an issue to document the feature more prominent 🚀