Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change in instrumentation behavior after 5.2.6 upgrade #4149

Closed
fixr opened this issue Apr 16, 2019 · 2 comments
Closed

Change in instrumentation behavior after 5.2.6 upgrade #4149

fixr opened this issue Apr 16, 2019 · 2 comments

Comments

@fixr
Copy link

fixr commented Apr 16, 2019

Ruby version: 2.5.5
Sidekiq / Pro / Enterprise version(s): Sidekiq Enterprise 5.2.6

After upgrading to 5.2.6 we noticed an unexpected behaviour in our metrics for one of queues. Right after the upgrade, queue delays went up to hours, which didn't reflect the reality on the enqueued items.

To calculate the delay, we use this formula:

queueing_delay_ms = (Time.current.to_f * 1000).to_i - (msg["enqueued_at"].to_f * 1000).to_i

The peculiarity on this queue is that we schedule jobs for many days in the future under it. It also has a higher chance of jobs failing and being retried.

We believe the changes introduced in cfe53e5 to be culprit. Before these changes, we would get a new enqueued_at value when it was re-enqueued. Now, the original value is kept.

Is this side-effect intended? If so, then we may look to alternatives. Any hint on how to handle it now, would be greatly appreciated!

mperham added a commit that referenced this issue Apr 16, 2019
@mperham
Copy link
Collaborator

mperham commented Apr 16, 2019

Not intended. I wanted to test an aspect of the Web UI and didn't realize this would break other cases. Thank you for reporting!

@fixr
Copy link
Author

fixr commented Apr 16, 2019

Thanks for the quick response and looking into it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants