LEAK: ByteBuf.release() was not called before it's garbage-collected. #3127

subham611 · 2024-04-02T12:45:38Z

Expected Behavior

I am using spring boot 2.6.13 version with spring-boot-starter-webflux dependency.
During low request count spring server works fine but when rps increased I am facing issue related to BUFFER LEAK.

Actual Behavior

Getting below mentioned error:
LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information. Recent access records: Created at:io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:403) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179) io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:116) org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:71) org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39) org.springframework.core.codec.CharSequenceEncoder.encodeValue(CharSequenceEncoder.java:91) org.springframework.core.codec.CharSequenceEncoder.lambda$encode$0(CharSequenceEncoder.java:75) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171) reactor.core.publisher.MonoSingle$SingleSubscriber.doOnRequest(MonoSingle.java:103) reactor.core.publisher.Operators$MonoInnerProducerBase.request(Operators.java:2731) reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194) reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068) reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:115) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96) reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68) reactor.core.publisher.Mono.subscribe(Mono.java:4455) reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:200) reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53) reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:251) reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:336) reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) reactor.core.publisher.MonoZip$ZipInner.onSubscribe(MonoZip.java:325) reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) reactor.core.publisher.Mono.subscribe(Mono.java:4455) reactor.core.publisher.MonoZip.subscribe(MonoZip.java:128) reactor.core.publisher.Mono.subscribe(Mono.java:4455) reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onError(FluxOnAssembly.java:544) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121) reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192) reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121) reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192) reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259) reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234) reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onError(MonoIgnoreElements.java:84) reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onError(FluxConcatArray.java:207) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:142) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) reactor.core.publisher.FluxPeekFuseable$PeekFuseableConditionalSubscriber.onNext(FluxPeekFuseable.java:503) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoCallable.subscribe(MonoCallable.java:62) reactor.core.publisher.Mono.subscribe(Mono.java:4455) reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:258) reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:78) reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.request(FluxContextWrite.java:136) reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onSubscribe(FluxContextWrite.java:101) reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2664) reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152) reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2400) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171) reactor.core.publisher.MonoSingle$SingleSubscriber.doOnRequest(MonoSingle.java:103) reactor.core.publisher.Operators$MonoInnerProducerBase.request(Operators.java:2731) reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194) reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068) reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:115) reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96) reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68) reactor.core.publisher.Mono.subscribe(Mono.java:4455) reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:200) reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53) reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.complete(MonoIgnoreThen.java:292) reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onNext(MonoIgnoreThen.java:187) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:251) reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:336) reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180) reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:101) reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:299) reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:337) reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:160) reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144) reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260) reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144) reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:400) reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:419) reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:603) reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:113) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) reactor.netty.http.server.HttpTrafficHandler.channelRead(HttpTrafficHandler.java:266) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:336) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:308) io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

JVM version (java -version): 17
spring boot version: 2.6.13

The text was updated successfully, but these errors were encountered:

violetagg · 2024-04-03T06:58:45Z

@subham611 Please follow this FAQ and provide the extended logs
https://projectreactor.io/docs/netty/release/reference/index.html#faq.memory-leaks

violetagg · 2024-04-03T07:00:41Z

@subham611 Consider checking Spring Boot supported versions
https://spring.io/projects/spring-boot#support

github-actions · 2024-04-11T06:26:25Z

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

subham611 · 2024-04-15T09:47:58Z

I updated spring boot to latest version 3.2.4 still getting this error.
I have added following jvm parameters, and not seeing any additional log

-XX:MaxDirectMemorySize=1024M
-Dio.netty.leakDetection.targetRecords=40
-Dio.netty.leakDetection.level=paranoid
-Dlogging.level._reactor.netty.channel.LeakDetection=debug

Do I need to set logging.level._reactor.netty.channel.LeakDetection=debug in application.properties.

violetagg · 2024-04-15T09:59:09Z

@subham611 Try to set it in application.properties

subham611 · 2024-04-15T10:00:53Z

yeah, saw this #1874 (comment)
Setting following properties in application.properties file

spring.netty.leak-detection=paranoid
logging.level._reactor.netty.channel.LeakDetection=DEBUG

subham611 · 2024-04-15T11:47:44Z

@violetagg I set above mentioned spring properties but not seeing any change in log message

{"timestamp":1713180562148,"thread":"reactor-http-epoll-1","logger":"io.netty.util.ResourceLeakDetector","message":"LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:404)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
	io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:116)
	org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:72)
	org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
	org.springframework.core.codec.CharSequenceEncoder.encodeValue(CharSequenceEncoder.java:90)
	org.springframework.core.codec.CharSequenceEncoder.lambda$encode$0(CharSequenceEncoder.java:74)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
	reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
	reactor.core.publisher.MonoSingle$SingleSubscriber.doOnRequest(MonoSingle.java:103)
	reactor.core.publisher.Operators$MonoInnerProducerBase.request(Operators.java:2909)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241)
	reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:115)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
	reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68)
	reactor.core.publisher.Mono.subscribe(Mono.java:4568)
	reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:202)
	reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
	reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:63)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
	reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:158)
	reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)
	reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)
	reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)
	reactor.core.publisher.MonoZip$ZipInner.onSubscribe(MonoZip.java:470)
	reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
	reactor.core.publisher.MonoZip$ZipCoordinator.request(MonoZip.java:220)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
	reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onSubscribe(FluxOnErrorResume.java:74)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117)
	reactor.core.publisher.MonoZip.subscribe(MonoZip.java:129)
	reactor.core.publisher.Mono.subscribe(Mono.java:4568)
	reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
	reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onError(FluxOnAssembly.java:544)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:241)
	reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:315)
	reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:241)
	reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:315)
	reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121)
	reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:241)
	reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:315)
	reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234)
	reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258)
	reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onError(MonoIgnoreElements.java:84)
	reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onError(FluxConcatArray.java:208)
	reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:241)
	reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:315)
	reactor.netty.FutureMono$FutureSubscription.operationComplete(FutureMono.java:193)
	io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
	io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
	io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
	io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
	io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
	io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
	io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
	io.netty.util.concurrent.PromiseCombiner.tryPromise(PromiseCombiner.java:170)
	io.netty.util.concurrent.PromiseCombiner.access$600(PromiseCombiner.java:35)
	io.netty.util.concurrent.PromiseCombiner$1.operationComplete0(PromiseCombiner.java:62)
	io.netty.util.concurrent.PromiseCombiner$1.operationComplete(PromiseCombiner.java:44)
	io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
	io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
	io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
	io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
	io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
	io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
	io.netty.util.internal.PromiseNotificationUtil.tryFailure(PromiseNotificationUtil.java:64)
	io.netty.channel.ChannelOutboundBuffer.safeFail(ChannelOutboundBuffer.java:754)
	io.netty.channel.ChannelOutboundBuffer.remove0(ChannelOutboundBuffer.java:339)
	io.netty.channel.ChannelOutboundBuffer.failFlushed(ChannelOutboundBuffer.java:691)
	io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:735)
	io.netty.channel.AbstractChannel$AbstractUnsafe.handleWriteError(AbstractChannel.java:950)
	io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:933)
	io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:570)
	io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
	io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
	io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

violetagg · 2024-04-15T12:01:05Z

@subham611 It doesn't seem that the changes are applied.

violetagg · 2024-04-15T12:02:13Z

You should be able to see output similar to the one here #1874 (comment)

subham611 · 2024-04-15T14:07:54Z

Is there any other way to set netty property?

violetagg · 2024-04-15T14:16:48Z

@subham611 Please consult with the reference documentation of the technologies that you use. For Spring Boot - it should be application.properties (there might be different versions of this - for testing, production etc), then the environment where you run your application - whether the provided JVM OPTS are applied etc.

I cannot help without knowing the environment where you run your app.

You should be able to check the exact JVM OPTS that are used to run your app and the binaries that are published.

subham611 · 2024-04-15T17:15:45Z

I have added this flag -Dlogging.level.io.netty.util.ResourceLeakDetector=debug as well and i do see following logs

{"timestamp":1713191024585,"thread":"main","logger":"io.netty.util.ResourceLeakDetector","message":"-Dio.netty.leakDetection.level: paranoid","context":"default","log_level":"DEBUG","is_logging_enabled":false}
{"timestamp":1713191024585,"thread":"main","logger":"io.netty.util.ResourceLeakDetector","message":"-Dio.netty.leakDetection.targetRecords: 40","context":"default","log_level":"DEBUG","is_logging_enabled":false}

Which confirms that leakDetection level is set to paranoid but somehow I am not seeing full logs during error.

I have setup a rate limiter using webFilter and I am suspecting some issue there.

@Component
public class RateLimiterFilter implements WebFilter {
  private final RateLimiter rateLimiter;
  @Override
    public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
       if(!rateLimiter.acquirePermission()) {
           log.warn("Too many requests");
           exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
           return Mono.empty(); // I am suspecting this might be causing some issue. 
       }
       return chain.filter(exchange);
    }
}

Let me remove this and try. ref: #518 (comment)

github-actions · 2024-04-23T06:27:45Z

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

subham611 · 2024-04-23T06:40:44Z

Above change didn't help.
I tried setting ResourceLeakDetector property programmatically, that didn't help either

 @Bean
    public void nettyServerCustomizer(){
        ResourceLeakDetector.setLevel(ResourceLeakDetector.Level.PARANOID);
        if(ResourceLeakDetector.isEnabled()) {
            System.out.println(ResourceLeakDetector.getLevel());
        }
    }

violetagg · 2024-04-23T07:00:32Z

@subham611 Please provide a reproducible example so that we can investigate further.

github-actions · 2024-05-02T06:26:46Z

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

github-actions · 2024-05-09T06:27:24Z

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open.

subham611 added status/need-triage A new issue that still need to be evaluated as a whole type/bug A general bug labels Apr 2, 2024

violetagg added for/user-attention This issue needs user attention (feedback, rework, etc...) and removed status/need-triage A new issue that still need to be evaluated as a whole labels Apr 3, 2024

violetagg self-assigned this Apr 3, 2024

github-actions bot added the status/need-feedback label Apr 11, 2024

github-actions bot removed the status/need-feedback label Apr 16, 2024

github-actions bot added the status/need-feedback label Apr 23, 2024

github-actions bot removed the status/need-feedback label Apr 24, 2024

github-actions bot added the status/need-feedback label May 2, 2024

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2024

violetagg removed type/bug A general bug for/user-attention This issue needs user attention (feedback, rework, etc...) labels May 9, 2024

violetagg added status/cannot-reproduce We cannot reproduce this issue and removed status/need-feedback labels May 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LEAK: ByteBuf.release() was not called before it's garbage-collected. #3127

LEAK: ByteBuf.release() was not called before it's garbage-collected. #3127

subham611 commented Apr 2, 2024

violetagg commented Apr 3, 2024

violetagg commented Apr 3, 2024

github-actions bot commented Apr 11, 2024

subham611 commented Apr 15, 2024 •

edited

violetagg commented Apr 15, 2024

subham611 commented Apr 15, 2024 •

edited

subham611 commented Apr 15, 2024

violetagg commented Apr 15, 2024

violetagg commented Apr 15, 2024

subham611 commented Apr 15, 2024

violetagg commented Apr 15, 2024 •

edited

subham611 commented Apr 15, 2024 •

edited

github-actions bot commented Apr 23, 2024

subham611 commented Apr 23, 2024

violetagg commented Apr 23, 2024

github-actions bot commented May 2, 2024

github-actions bot commented May 9, 2024

LEAK: ByteBuf.release() was not called before it's garbage-collected. #3127

LEAK: ByteBuf.release() was not called before it's garbage-collected. #3127

Comments

subham611 commented Apr 2, 2024

Expected Behavior

Actual Behavior

violetagg commented Apr 3, 2024

violetagg commented Apr 3, 2024

github-actions bot commented Apr 11, 2024

subham611 commented Apr 15, 2024 • edited

violetagg commented Apr 15, 2024

subham611 commented Apr 15, 2024 • edited

subham611 commented Apr 15, 2024

violetagg commented Apr 15, 2024

violetagg commented Apr 15, 2024

subham611 commented Apr 15, 2024

violetagg commented Apr 15, 2024 • edited

subham611 commented Apr 15, 2024 • edited

github-actions bot commented Apr 23, 2024

subham611 commented Apr 23, 2024

violetagg commented Apr 23, 2024

github-actions bot commented May 2, 2024

github-actions bot commented May 9, 2024

subham611 commented Apr 15, 2024 •

edited

subham611 commented Apr 15, 2024 •

edited

violetagg commented Apr 15, 2024 •

edited

subham611 commented Apr 15, 2024 •

edited