Queued pull consumers with WorkQueue retention

[bounoable]

Hey, I'm currently working on an event-sourcing library that uses NATS as the primary backend for the event bus implementations but I'm struggling with the implementation of the JetStream driver. Users of the library should be able to configure to either use NATS Core, JetStream with push consumers, or JetStream with pull consumers when creating the event bus. The first two scenarios work out but I can't get queues to work with pull consumers like it's mentioned here and here because of a limitation of the WorkQueue retention policy.

Here's how I tried to design the "JetStream Pull Consumer" event bus:

• The event bus creates a single stream with the wildcard subject *. All consumers refer to this stream.
• Users can enable durable subscriptions via an option
• Users can enable queue groups via an option
• Users can enable "pull" mode via an option
• If the user enables queues in "pull" mode, the retention policy is set to WorkQueue to shim the load-balancing behavior of queue group in "push" mode
• For each event that is subscribed to, a consumer is created – either ephemeral or durable (if the user enabled durable subscriptions).
• Subsequent subscriptions to the same event are handled internally, so that a single subject is not subscribed to twice by a service (instance)

A consumer is created for each event to support load-balancing between replicas of a (micro)service, so that
consumer name = service name + event name and filter subject = event name

Problem is that the WorkQueue policy doesn't allow multiple consumers to have the same filter subjects. This means that, using this design, there can be no two different services that subscribe to the same event (but there can be two instances of the same service that subscribe to the same event because they share a consumer).

Stream per event

I also tried to create one stream per event and then – for each event that a service subscribes to – create a consumer without a filter subject. Problem here is that the library needs to support wildcard subscriptions on all possible events using the * wildcard, but a single consumer cannot return messages of multiple streams.

---

So, anyone has an idea how this could be implemented? Thanks in advance.

[derekcollison]

A workqueue binds a stream to a single consumer and semantic. You can bind as may pull subscribers as you want to that consumer however for horizontal scalability.

If you have a mixed mode semantics for a stream and want multiple and mutually exclusive consumers, try the interest retention policy.