New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory Leak in reactor-netty when using Spring WebClient with blocked flux. Issue: Databuffer is not released. #3178
Comments
@KrishnakanthYachareni From the configuration I don't see any changes related to the memory management, which means that you are running with Pooled ByteBufs. Is it possible that you see this reserved memory? |
@violetagg Yes, my application has been using default Pooled ByteBus, as I did not have any manual configuration related to mememory management. You can see below it is a reserved memory. Why does ByteBuf memory is not clearing, is the following snippet not releasing the buffer properly? Flux<FcmResponseWrapper> response = fcmRestClient.sendPush(fluxRequests, appName, eventIdentifiers);
List<Tuple2<Long, FcmResponseWrapper>> listOfSendResponses = response.elapsed()
.collectList()
.block(); // Thread blocking. |
Is it possible to enable memory metrics provided by Reactor Netty |
@violetagg Just to clarify that in my code no other objects are allocating more memory and memeory is leaking from reactor related objects. That I verfied during JVM profiling. bytebuf are not emitted but ConnectionProvider metrics are emitting. Following code used to enable the reactor metrics. am I missing anything?
private ConnectionProvider getConnectionProvider() {
return ConnectionProvider.builder("fcmConnection")
.metrics(true)
.build();
}
@Bean
public NettyServerCustomizer nettyServerCustomizer(){
return httpServer -> httpServer.metrics(true, Function.identity());
} My suspect is messages.flatMap(msgPair -> webClient.post()
.uri(appURI)
.header(HttpHeaders.AUTHORIZATION, accessToken)
.body(Mono.just(msgPair.getFirst()), FirebaseRequest.class)
.exchangeToFlux(clientResponse -> this.handleResponse(clientResponse, msgPair.getSecond())))
.subscribeOn(Schedulers.fromExecutor(executor)); |
Add this to the client |
@violetagg I collected the byteBuff memory metric and both seems always constant and same value. |
@KrishnakanthYachareni You need the
vs
|
@violetagg My bad, I miss understood the mertics name. Both memories are most of the time showing as Note: Just to mention Non-Heap memory is gradually increasing and not going down. |
@KrishnakanthYachareni So to summarise the active memory is not increasing constantly but it returns to 0? |
@violetagg Yes. that is right! |
@KrishnakanthYachareni So looks like no memory leaks? |
@violetagg I did these changes, this could be a reason memory leak got fixed. From webClient.subscribeOn(Schedulers.fromExecutor(executor)); To webClient.publishOn(Schedulers.fromExecutor(executor)); |
@KrishnakanthYachareni I cannot comment on the change from |
@violetagg yes please, you can close it. |
My service is responsible to consume the events from Kafka and will make a HTTP class to external server (REST API).
Expected Behavior
There shouldn't be memory leaks when making multiple HTTP calls to external service.
Actual Behavior
When I did the performance testing, heap memory is not cleared. GC is not able to collect due to reactor DataBuff is not relasing the data.Steps to Reproduce
I can't provide complete code to reproduce it but here's the main configuration.
WebClient Configuration
WebClient usage
How I'm calling the above method is:
I tried to get the debug logs of LeakDetection but I couldn't find them in the logs
Possible Solution
Your Environment
Spring boot
2.7.12
netty
, ...):java -version
): Java 11uname -a
):The text was updated successfully, but these errors were encountered: