New Redis API, too many subscriber timeout

[Plawn]

Describe the bug

edited -> see reproducer

Expected behavior

We are subscribed

Actual behavior

Error

How to Reproduce?

Create a quarkus app with version 2.11.1.Final
Use the given files

Output of uname -a or ver

No response

Output of java -version

11

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of mvnw --version or gradlew --version)

maven 3.8

Additional information

No response

[quarkus-bot]

/cc @cescoffier, @gsmet, @machi1990

[Plawn]

Hello, after even more debugging I found out, that the issue is having too many listeners, or at list the issue appears when i have 5 or more listeners

[gsmet]

Better provide a full reproducer in the form of a small Maven project.

[Plawn]

yes will do 👍

[Plawn]

Hello,
Here is the reproducer

https://github.com/Plawn/redis-quarkus-reproducer

here is the main test file

https://github.com/Plawn/redis-quarkus-reproducer/blob/master/src/main/java/org/acme/HttpCacheProvider.java

To test: just mvn quarkus:dev

When having too many listener there is a timeout

More over, the listener can't start without the @startup annotation

[cescoffier]

This is kind of expected. Subscribers need to hold a connection to the Redis server (that's the Redis protocol). However, by default, the connection pool is set to have only six connections.

So you should restructure your application to reduce the number of connections (use a single subscriber listening to multiple keys) or increase the number of connections:

quarkus.redis.max-pool-size=....

[Plawn]

Ok I see, in the redis documentation it stated that up to 10 000 listeners could listen to the same channel, so I did not expect this kind of issue.
Thanks !
However is it expected to only be able to start listening at the start of the Quarkus app ? If I'm not listening inside a Bean annotated with @startup the subscription won't work because of a null context

Edit: Changing the pool size solved my issue, but I still wonder why I only works at startup

[cescoffier]

Can you give more details about this startup issue?
Do you mean you can only register subscribers during the startup of the application?

[Plawn]

Hello !

Yes, If I don't mark the bean creating the redis subscriber with @startup (like in the example), I will have a vertx null context exception.

[cescoffier]

Can you provide a reproducer? The Redis data source requires Vert.x but the Vert.x instance is created quite early in the process.

[cescoffier]

I believe I reproduced it:

2022-08-18 11:25:20,674 WARN  [io.net.uti.con.AbstractEventExecutor] (vert.x-eventloop-thread-11) A task raised an exception. Task: io.vertx.core.impl.EventLoopContext$$Lambda$1042/0x00000008007fef90@1dac7cf9: java.lang.IllegalArgumentException: Parameter 'context' may not be null
        at io.smallrye.common.constraint.Assert.checkNotNullParamChecked(Assert.java:32)
        at io.smallrye.common.constraint.Assert.checkNotNullParam(Assert.java:26)
        at io.smallrye.common.vertx.VertxContext.isDuplicatedContext(VertxContext.java:114)
        at io.smallrye.common.vertx.VertxContext.getOrCreateDuplicatedContext(VertxContext.java:31)
        at io.quarkus.redis.runtime.datasource.ReactivePubSubCommandsImpl$AbstractRedisSubscriber.lambda$subscribe$2(ReactivePubSubCommandsImpl.java:149)

[cescoffier]

(and it's likely a bug)

[Plawn]

I believe I reproduced it:

2022-08-18 11:25:20,674 WARN  [io.net.uti.con.AbstractEventExecutor] (vert.x-eventloop-thread-11) A task raised an exception. Task: io.vertx.core.impl.EventLoopContext$$Lambda$1042/0x00000008007fef90@1dac7cf9: java.lang.IllegalArgumentException: Parameter 'context' may not be null
        at io.smallrye.common.constraint.Assert.checkNotNullParamChecked(Assert.java:32)
        at io.smallrye.common.constraint.Assert.checkNotNullParam(Assert.java:26)
        at io.smallrye.common.vertx.VertxContext.isDuplicatedContext(VertxContext.java:114)
        at io.smallrye.common.vertx.VertxContext.getOrCreateDuplicatedContext(VertxContext.java:31)
        at io.quarkus.redis.runtime.datasource.ReactivePubSubCommandsImpl$AbstractRedisSubscriber.lambda$subscribe$2(ReactivePubSubCommandsImpl.java:149)

Yes I exactly got this error !

[cescoffier]

@Plawn see #27361.

Closing this issue, as our discussion is about another issue. Please use #27361 to discuss that other issue.