Netty occasional high latency (+500ms) observed even for a single endpoint in Event loop

[swasti7]

I am using AHC client built on top of Netty 4. Occasionally 1-20% of the time, latency increases by 100-500ms for a 40ms endpoint. I am 100% sure that response time of endpoint itself has not increased but this is a characteristic of Netty itself.

This is how I am running it. I create a client on main thread at the start (one client only) and test HTTPs endpoint every 10 seconds i.e every 10 second there is a single endpoint which Netty with a thread pool of 1 thread asynchronously process.
Main thread sleeps for 10 seconds and submit another https request to the client.

For a 40ms endpoints, 80% of the time I see expected time to first byte i.e 40-50ms.
Rest of the time, It varies from 100ms - 700ms.

Any idea what could be happening? Also the first request after the client is created has very high latency. I initially assumed first request is expected to have high latency. But it keeps coming back intermittently couple of times.

[normanmaurer]

@swasti7 I think you might want to ask on the AHC group. One thing you could also check is if you might block the EventLoop

[slandelle]

Sadly, AHC is a dead project. I stepped down as a committer several years ago and the person who took over actually never really did and contributions never went flowing.

[swasti7]

Thank you for responding @normanmaurer. I only have AHC event handlers on the request which just records the time spent since the start of the request for example onStatusReceived event handler. I don't see any reason why Event loop could be blocked. Is there any way to check what is going on with the Event loop ?

AHC is not actively maintained so not sure I will get a response there. I did a quick prototype to find out TPS i can achieve with Netty for HTTPS requests with minimal (few milliseconds) latency at max.
Do you know if HTTPS requests on Netty can have these kind of variances ??

[swasti7]

@slandelle Yeah thats why i don't think I will get a response on AHC group. There is no active maintainer since a long time. I plan to use Netty directly eventually. But used AHC for quick prototyping and running tests to understand TPS that can be achieved for HTTPS requests with minimal latency jitters.

[franz1981]

If you can reproduce it, why not trying to profile it?
You can use async profiler both CPU and LOCK events
Note: consider that Netty pooled allocator can block an event loop thread if the allocation rate exceed the thread local capacity AND the allocation happen on a contended arena

[swasti7]

Thanks for your response @franz1981.
When you say "Netty pooled allocator can block an event loop thread if the allocation rate exceed the thread local capacity ". You mean that if number of channels exceed thread capacity then it can block event loop thread?

This should not happen when there is just one request every 10 seconds right ? One task should not exceed thread local capacity ?

Are you talking about this profiler?

[franz1981]

You mean that if number of channels exceed thread capacity then it can block event loop thread?

I mean when the allocation pressure (ie amount of pooled ByteBuf allocated) exceed the available capacity in the thread-local allocation buffer of the pooled allocator, the allocator will force allocating from the (potentially) shared arenas, that can compete each others doing it.
Yes, the profiler is right and you can use JFR as well with lock-profiling to find this out easily.