Variabilizing the jitter for sidekiq retries

[jsagl]

Hi,

First of all, thank you for your work in developing and maintaining Sidekiq.

I know the topic has been discussed in a few issues, notably the one pointed at in the documentation (#480), and understand the usecase for the jitter in the retries, but was wondering if you'd be open to variabilizing it.

I am well aware that sidekiq is not meant to schedule jobs precisely, but I feel like there is a big difference between "not precisely scheduling" and preventing a retry within less than 15 seconds, or a roughly regular retry. We are using sidekiq to poll the status of transactions on the blockchain, and saw results we did not expect when trying to poll every 3 seconds (more or less, considering the polling interval) using the sidekiq_retry_in block. Hence we found the jitter and read about why it was implemented. While it makes perfect sense to have one, I believe it's not a solution suiting every business needs, and offering the possibility to remove or reduce it on a per job basis would be very helpful.

I have monkey patched it quite easily, adjusting the JobRetry#process_retry and JobRetry#delay_for, but was wondering if you'd be open to a PR so that everyone can enjoy / benefit from the feature.

Cheers

[mperham]

You can drop the average scheduled poll time in order to get more accurate job scheduling:

Sidekiq.configure_server do |config|
  config[: average_scheduled_poll_interval] = 3
end

but I'm not interested in reducing the jitter. That's really necessary to smear out large blocks of failed jobs. Maybe instead of retrying with its jitter, you should catch the error and reschedule.

[jsagl]

Thank you for your answer. Performance wise, is there any difference between rescheduling and raising/retrying ? I do need to use the retry_count and benefitted from the one available in the sidekiq_retry_in block. I can probably pass this info as argument to the job I''ll reschedule, but I am not willing to do this unless I see a real performance benefit.

[mperham]

The performance diff is no jitter.