Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flooding of 'Failed to check/redeclare auto-delete queue(s)' error messages #1477

Closed
oosterholt opened this issue Jul 27, 2022 · 2 comments · Fixed by #1523
Closed

Flooding of 'Failed to check/redeclare auto-delete queue(s)' error messages #1477

oosterholt opened this issue Jul 27, 2022 · 2 comments · Fixed by #1523
Labels
backport:2.4.x (obsolete) type: enhancement
Milestone
3.0.0

Comments

@oosterholt
Copy link

oosterholt commented Jul 27, 2022 •
edited

In what version(s) of Spring AMQP are you seeing this issue?
2.2.20 up to latest (probably earlier as well).

Describe the bug
When RabbitMQ service restarts, our log is flooded with error messages when simply trying to recreate queues.
Once RabbitMQ is started again, the queues get re-created correctly and the logging stops.

ERROR 2022-07-27 08:33:36 (org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer) Failed to check/redeclare auto-delete queue(s).
org.springframework.amqp.AmqpIOException: java.io.IOException
    at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:70) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:602) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:725) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:252) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2180) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2153) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2133) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueInfo(RabbitAdmin.java:463) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:447) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.attemptDeclarations(AbstractMessageListenerContainer.java:1930) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.redeclareElementsIfNecessary(AbstractMessageListenerContainer.java:1911) [spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.initialize(SimpleMessageListenerContainer.java:1375) [spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1221) [spring-rabbit.jar:2.4.6]
    at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: java.io.IOException
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:129) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:125) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:396) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1225) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1173) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1342) ~[amqp-client.jar:5.15.0]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.connectHostPort(AbstractConnectionFactory.java:653) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.connect(AbstractConnectionFactory.java:618) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:565) ~[spring-rabbit.jar:2.4.6]
    ... 12 more
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:502) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:326) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1225) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1173) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1342) ~[amqp-client.jar:5.15.0]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.connectHostPort(AbstractConnectionFactory.java:653) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.connect(AbstractConnectionFactory.java:618) ~[spring-rabbit.jar:2.4.6]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:565) ~[spring-rabbit.jar:2.4.6]
    ... 12 more
Caused by: java.net.SocketException: Connection reset
    at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323) ~[?:?]
    at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350) ~[?:?]
    at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803) ~[?:?]
    at java.net.Socket$SocketInputStream.read(Socket.java:966) ~[?:?]
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:244) ~[?:?]
    at java.io.BufferedInputStream.read(BufferedInputStream.java:263) ~[?:?]
    at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:294) ~[?:?]
    at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:184) ~[amqp-client.jar:5.15.0]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:665) ~[amqp-client.jar:5.15.0]
    ... 1 more

To Reproduce
Either have @Bean declared queues or queues declared via RabbitAdmin.
Start application
Restart RabbitMQ service

Expected behavior
Do not log error messages when spring AMQP simply is retrying (using configured BackOff polici) to restore declared queues.
Or at least change the log level to info or debug.
When to queues cannot be restored (after number of tries or timeout) an error message is needed.
@garyrussell
Copy link
Contributor

We can remove the stack trace, but I believe that the logs must remain, perhaps at WARN level which would allow you to suppress them.

Please note that 2.2.x is no longer supported as OSS https://spring.io/projects/spring-amqp#support

@garyrussell garyrussell added this to the Backlog milestone Jul 27, 2022
@oosterholt
Copy link
Author

Thanks for the quick reply.
WARN level would indeed help.

@garyrussell garyrussell modified the milestones: Backlog, 3.0.0 Oct 20, 2022
garyrussell added a commit to garyrussell/spring-amqp that referenced this issue Oct 20, 2022
@garyrussell
spring-projectsGH-1477: Reduce Log Noise While Broker Down
848d4b2 
Resolves spring-projects#1477

Only log the stack trace on the first failure.

Tested with a Boot app and stop/start/stop the broker.

**cherry-pick to 2.4.x**
@garyrussell garyrussell mentioned this issue Oct 20, 2022
Merged
artembilan pushed a commit that referenced this issue Oct 21, 2022
@garyrussell @artembilan
GH-1477: Reduce Log Noise While Broker Down
77dfa65 
Resolves #1477

Only log the stack trace on the first failure.

Tested with a Boot app and stop/start/stop the broker.

**cherry-pick to 2.4.x**
artembilan pushed a commit that referenced this issue Oct 21, 2022
@garyrussell @artembilan
GH-1477: Reduce Log Noise While Broker Down
9242967 
Resolves #1477

Only log the stack trace on the first failure.

Tested with a Boot app and stop/start/stop the broker.

**cherry-pick to 2.4.x**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
backport:2.4.x (obsolete) type: enhancement
Projects
None yet
Milestone
3.0.0
Development

Successfully merging a pull request may close this issue.

GH-1477: Reduce Log Noise While Broker Down garyrussell/spring-amqp
2 participants
@garyrussell @oosterholt