How to clean out these Busy jobs.

[resistorsoftware]

Ruby version: 3.1.1
Rails version: 7.0
Sidekiq / Pro / Enterprise version(s): 6.4.1

My initializer for Sidekiq is:

require "sidekiq"

Sidekiq.configure_client do |config|
  config.redis = {size: 1}
end

 Sidekiq.configure_server do |config|
   config.redis = {size: 12}
 end

This is running on 2 Dynos at Heroku with the procfile: worker: bundle exec sidekiq -c 3

I have no idea what I have done, but I did it. I have 3 jobs in busy status. I cannot kill them. I cannot seemingly do anything. I am stuck. My dashboard shows this:

No matter what I try, these things persist. What the heck do I do to kill these things? I started them just at my heroku command line using

 SalesReportJob.perform_later(params)

Thinking I could just monitor what happened.. instead these jobs are like zombies!!

[mperham]

Restarting Sidekiq will typically clear them out.

[resistorsoftware]

I ran a heroku ps:restart to document what happens. What happens is this...

workers boot... then I see:

Starting a Sales job for aldkjfslkdjfksldjfskjdf
Starting a Sales job for aldkjfslkdjfksldjfskjdf
Starting a Sales job for aldkjfslkdjfksldjfskjdf

WARN: Terminating 3 busy worker threads
2022-03-10T00:29:18.257012+00:00 app[worker.2]: pid=4 tid=3is WARN: Work still in progress [#<struct Sidekiq::BasicFetch::UnitOfWork queue="queue:default", j

INFO: Pushed 3 jobs back to Redis
2022-03-10T00:29:18.278272+00:00 app[worker.2]: E, [2022-03-09T16:29:18.278165 #4] ERROR -- : [ActiveJob] [SalesReportJob] [670cc0da-4cf4-4502-bc4f-a30fa2151c3c] Error performing SalesReportJob (Job ID: 670cc0da-4cf4-4502-bc4f-a30fa2151c3c) from Sidekiq(default) in 3866056.94ms: Sidekiq::Shutdown (Sidekiq::Shutdown):

So basically, these things just keep on coming back to life.. I cannot kill them!!

Maybe I am missing the basics. How to restart Sidekiq?

[mperham]

That's right. Sidekiq does its best to not lose jobs. Your jobs aren't finishing so you'll need to debug and figure out why. I'm not sure the best way to debug in production on Heroku.

[resistorsoftware]

So the only recourse, is to let these 3 jobs sit there forever? Wow. I guess I can always just delete Redis. That should do the trick.

[resistorsoftware]

Simply by fixing the job, they all finished. All clean now. Thanks!

[mperham]

Or deploy a fix. Or deploy a change that adds “return” to the top of the perform method. Or use the Queue API to clear all the jobs in that queue after shutting down Sidekiq.

…

On Wed, Mar 9, 2022 at 16:58 David Lazar ***@***.***> wrote: So the only recourse, is to let these 3 jobs sit there forever? Wow. I guess I can always just delete Redis. That should do the trick. — Reply to this email directly, view it on GitHub <#5237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAWX6YTAJ2XALDEMCXNBDU7FCLVANCNFSM5QLB3YWA> . You are receiving this because you commented.Message ID: ***@***.***>