With the release of 7.0 and Capsules, will sidekiq.yml support capsule creation/configuration?

[BobbyMcWho]

We run Enterprise & Sidekiqswarm, with 3 separate configs for different queue sets. I want to run a large async searchkick reindex, and the number of jobs will be in the thousands generated all at once by searchkick. Rather than saturate all of our 'general' processes/threads w/ them, I'd like to reduce the concurrency. My initial thought is to just use sidekiq-ent concurrent rate limiter, but that will still saturate the threads until the locks/retry scheduling shakes itself out.

Sidekiq 7 w/ capsules sounds like a great solution, but I am struggling to figure out how I can configure the capsule on only 1 of 3 server configs (since we pass them as file configs).

[mperham]

Today, sidekiq.yml and -C only work with the default capsule. Other capsules must be configured in Ruby.

[nateberkopec]

Yes. #5547 💥

[mperham]

Nothing is stopping you from doing that, I just don't want to document it.

[bert-mccutchen]

Right, I understand that. You said that you don't want to document it because you don't endorse it.

However, you haven't given a reason as to why you don't endorse it. Which was the primary concern with my last post, everything else was to back up my position as to why I believe it should be a documented feature.

[mperham]

I don't particularly like YAML as a config language, it's easy to get bugs with bad indentation.

Personally I would never use sidekiq.yml. I'd use the CLI arguments (-e staging -c NN) or Ruby.

[bert-mccutchen]

So then why document the usage of the YAML file at all? Also, why have features documented, such as the global max retries, that can only be configured via the Sidekiq YAML?

The capability to configure Capsules in YAML is very useful information. It would be more beneficial to the public to not withhold such useful information based on personal preference. Especially when there is already a precedence of documenting the usage of YAML in this open source project.

[mperham]

@bert-mccutchen Backwards compatibility and legacy.