High latency observed with Sinatra API intermittently

[imnani]

We are observing high latencies intermittently with our APIs which use the Sinatra framework and puma web server(single mode). We don't see any issues/delays with puma worker threads accepting incoming requests and processing the requests. The issue primarily lies with the delay involved in returning a response from the Sinatra controller. We use New Relic for our observability needs and it clearly indicates that the latency is caused by the controller.
Rack acts as an interface between the Sinatra framework and the puma server and I suspect rack could be contributing to this issue too. We are observing this intermittent issue only during peak load. Any guidance could be appreciated.

Library Versions:
Sinatra: 2.2.2
Puma: 5.5.2
Rack: 2.24

[dentarg]

What is your controller doing? 😄 I would concentrate on that 😉

[imnani]

If you observe the new relic trace it indicates that most time is consumed for the process Application code in (Nested/Controller/Sinatra/V2::OrderController/POST (unknown)). As I mentioned earlier based on service logs we figured out that at the controller level logic processing was pretty fast but when we returned the response from the controller it took a while for it to reach to APIGEE layer.

[dentarg]

APIGEE is a remote service, right? Isn't it just the time it takes to communicate with it?

Sinatra does not add 17 seconds of overhead

[jkowens]

Beside taking another look at the application code, maybe look at the Apigee and nginx logs.

[imnani]

New Relic instrumentation is done only for our application code and from New Relic APM it is pretty evident that 17 seconds overhead is added at the Sinatra controller level.

[dentarg]

I think you need to figure out this one yourselves. Sinatra is open-source after all, so you have that going for you.