Makara and sidekiq mysql in use

[grossjo]

Ruby version: 2.0
Sidekiq / Pro / Enterprise version(s): Sidekiq 4.1.3
makara: 0.3.6

I've recently started integrating the makara gem, and noticed some strange behaviour.

ActiveRecord::StatementInvalid: Mysql2::Error: This connection is in use by:

An idea what could be causing this?

[simonmorley]

I've had to stop using Maraka with Sidekiq completely. Not sure if there's a thread issue or wot yet...

Whole application came to a grinding halt, page faults hitting 1,000/s. Memory over-loaded.

General misery. Will investigate further later...

[bleonard]

Sorry to hear that. We are not currently using Sidekiq. We've done some tests and it seemed ok, but the scale was more of a staging load. Will look into it when we get there.

[mnelson]

Do you guys have an establish_connection call in your Sidekiq.configure_server block? I'm not using sidekiq + makara right now but that was the first thing that came to mind.

[simonmorley]

Nope. Was that a ---> "you should add that" or.... you should not have that in your config??

rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = Rails.env || 'production'

require 'sidekiq'
require 'sidekiq-status'

sidekiq_config = YAML.load(ERB.new(File.read(rails_root + '/config/redis_server.yml')).result)

Sidekiq.configure_server do |config|
  config.redis = { :host => "#{sidekiq_config[rails_env]["write"]}"}
end

Sidekiq.configure_client do |config|
  config.redis = { :host => "#{sidekiq_config[rails_env]["write"]}"}
end

schedule_file = "config/scheduler.yml"

if File.exists?(schedule_file)
  Sidekiq::Cron::Job.load_from_hash! YAML.load_file(schedule_file)
end

I'm just confirming for sure that the issue WAS Makara ;)

We have significant load on our Sidekiq servers but the problem was easy to simulate, with limited traffic...

I'm going to try and test locally also again since we're running in Kubernetes.

[mnelson]

I believe you should have that in your server block. I essentially treat it the same as a unicorn / puma fork.

[simonmorley]

I'll whack that in today and report back with findings.

[paolodona]

Hi guys / @simonmorley
just checking if you have any findings / tips on how to configure Sidekiq to work well with Makara?

[jwg2s]

@simonmorley also curious to hear about this, we're attempting to use makara/sidekiq and would love to hear what you found

[bleonard]

Seems like maybe a duplicate of: #151