New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow sidekiq_retry_in to return a range #4957
Conversation
@mperham Any reason why? It would make it easier to reason about exponential backoffs with jitter. (Thank you for your work on sidekiq!) |
“No description provided.” if you make no effort to explain your PR to others, I make no effort to review it. |
@mperham Updated. I appreciate the fast feedback, but sometimes it's too fast. ;-) You closed it in 5 minutes while I was still pondering the best description. Please let me know if you want me to expand on this description. |
👍🏻 |
I appreciate the feedback. In the future I’ll give people a while to finalize the PR before taking any actions. |
What happens if the block returns nil? 0? |
Yes:
hence https://github.com/mperham/sidekiq/pull/4957/files#diff-a2a78bd5586e1300341bb91dc8be382aa98126dcde11bd507fe9886d2e643ee7R223 hence If you mean an endless range then it raises an exception:
I'm not sure what is the best behaviour here. |
Other reasons why I think this change makes sense:
|
Well I wonder then why let the user return a range but rather build jitter
into sidekiq directly. So if the user returns 30 we would actually jitter
it between 20 and 40 for instance.
…On Sun, Aug 8, 2021 at 01:17 maurycy ***@***.***> wrote:
Other reasons why I think this change makes sense:
- random retries should be encouraged, even seconds_to_delay already
uses rand,
- sidekiq_retry_in is always a bit probabilistic, because there is no
way to guarantee a retry exactly at a given time.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#4957 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAWX77THJYRPBCJOACT73T3Y4TBANCNFSM5BXZ6CWA>
.
|
Why not both? Some users might want to retry within a short period, others prefer a longer one, and Moving in the direction of building in explicit randomness needs some consideration: which jitter algorithm? leave the current seconds_to_delay formula as is and focus only on the I like the direction, though. What do you think? |
I strongly believe software needs less knobs, not more. I'd prefer the user return a single Integer and we add jitter internally. That's how Sidekiq's retry subsystem works today. Is the root issue here "sidekiq_retry_in should include jitter so lots of jobs don't pile up at the exact same time"? |
There's still a need for configuration I think.
Yes. Another idea: |
Ok, I think the simpler solution is to add jitter to the return value from sidekiq_retry_in. perform_in can't add jitter as people really depend on that timing but retries are fair game and already do. |
The idea is to make it easier to implement (and reason about) exponential (and not only!) backoffs with jitter.