Middleware

Sidekiq has a similar notion of middleware to Rack: these are small bits of code that can implement functionality. Sidekiq breaks middleware into client-side and server-side.

• Server-side middleware runs 'around' job processing. Sidekiq's retry feature is implemented as a simple middleware.
• Client-side middleware runs before the pushing of the job to Redis and allows you to modify/stop the job before it gets pushed. Client middleware may receive the class argument as a Class object or a String containing the name of the class.

Writing your own middleware is easy; this is the server-side middleware which ensures that ActiveRecord connections are closed after each job is processed:

class Sidekiq::Middleware::Server::ActiveRecord
  def call(worker, job, queue)
    yield
  ensure
    ::ActiveRecord::Base.clear_active_connections! if defined?(::ActiveRecord)
  end
end

Your middleware will be called with the worker instance which will process the job along with the full Hash which represents the job to process and the name of the queue it was pulled from.

class AcmeCo::MyMiddleware
  def initialize(options=nil)
    # options == { :foo => 1, :bar => 2 }
  end
  def call(worker, msg, queue)
    yield
  end
end

You then register your middleware as part of the chain:

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add AcmeCo::MyMiddleware, :foo => 1, :bar => 2
  end
end

I'd suggest putting this code in config/initializers/sidekiq.rb in your Rails app.

Remember that the jobs running in the Sidekiq server can themselves push new jobs to Sidekiq, thus acting as clients. You must configure your client middleware within the configure_server block also in that case:

class AcmeCo::MyClientMiddleware
  def call(worker_class, msg, queue, redis_pool)
    # return false/nil to stop the job from going to redis
    return false if queue != 'default'
    yield
  end
end

Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.add AcmeCo::MyClientMiddleware
  end
end
Sidekiq.configure_server do |config|
  config.client_middleware do |chain|
    chain.add AcmeCo::MyClientMiddleware
  end
  config.server_middleware do |chain|
    chain.add AcmeCo::MyMiddleware, :foo => 1, :bar => 2
  end
end

As of Sidekiq 3.0, client-side middleware takes a fourth option, redis_pool, which is a connection pool for the Redis instance where this job is going.

Default Middleware

By default, Sidekiq's server middleware includes the following:

def self.default_middleware
  Sidekiq::Middleware::Chain.new do |m|
    m.add Sidekiq::Middleware::Server::Logging
    m.add Sidekiq::Middleware::Server::RetryJobs
    m.add Sidekiq::Middleware::Server::ActiveRecord
  end
end

• Logging - logs the start and finish of message processing
• RetryJobs - puts the message in the retry queue if it raises an exception
• ActiveRecord - closes active connections associated with the worker thread

If you need to remove a middleware for some reason, you can do this in your configuration:

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.remove Sidekiq::Middleware::Server::RetryJobs
  end
end

Previous: API Next: Resque Compatibility