Skip to content

How this gem interacts with Sidekiq

David Guthu edited this page Mar 26, 2020 · 4 revisions

Sidekiq is actually quite simple. It consists of two parts, a client and a server. It is a little more complex than this but for the sake of this gem that should be sufficient information to wrap our heads around how this gem works.

Sidekiq::Client

When your worker class is adding a job to sidekiq it goes through the configured middleware before it is finally sent to redis. Depending on whether you told the worker to perform the job async or on a schedule it ends up either directly in the insert-queue-name queue or in the schedule queue. They are treated differently in Sidekiq but the way this gem works is the same for both scenarios.

When the client middleware runs we create a lock for a combination of things:

  • The class name of the worker that is running the job (can be skipped by configuration unique_across_workers)
  • The name of the queue (can be skipped by configuration unique_across_queues)
  • The arguments provided to the perform_async|perform_in OR using a filter method/proc of your choosing (see unique_args section in the readme.)

Based on the above we create a digest of what is left in the job hash for the keys class, queue, unique_args. If you are in doubt you can check the enqueued item in redis. (The easiest way to do it is to push it to the schedule queue and doing Sidekiq::ScheduledSet.new.each { |job| p job }.

Now the job is locked and no more items with the same combination of arguments will be allowed to be pushed to redis as there is already a lock created.

WARNING If you decide to use the lock_expiration configuration note that it is a dangerous option to use. It means the lock will expire exactly when you tell it to, regardless of if the job is running, didn't run or completed. In some situations it might be wanted like for a job that is unique: :while_executing and completes really fast.

Sidekiq::Processor

This is where the code in your perform method is executed. Before we execute the code however the server middleware is run. The locks behave differently but each lock class has an execute method and this is what gets called by the server middleware. If the execute method is successful it will yield to the server middleware. See the page Locking & Unlocking for more information on how the locks differ.