Skip to content

Commit

Permalink
refactor: check endpoint handler is async only once (#2536)
Browse files Browse the repository at this point in the history
* refactor: check endpoint handler is async only once

We improve the dispatch in the routing module to only check once whether the handler is async. This gives an improvement of 2.5% (sync), 1.82% (async) in the number of requests/s. The average latency decreased 1.6% (sync) and 1.5% (async).

Note that we had to use a cast in the helper function, as the typeguard does not work for the negative case. In the main branch the code is working without a cast, because the typeguard return type is in practice `AwaitableCAllable[Any]`, which end up swallowing the other types in the union.

Benchmarking
We use a simple json app, with both a sync and async endpoint, and the wrk tool to get the measurements.

The measuerements were done on a Macbook Pro with M1 chip, 16GB of memory and macOS 12.3. The Python version used for the tests is Python 3.12.2, and the uvicorn version is 0.27.1

Before
```
$ wrk http://localhost:8000/sync
Running 10s test @ http://localhost:8000/sync
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   733.77us   55.57us   3.21ms   78.35%
    Req/Sec     6.84k   147.06     7.15k    87.13%
  137474 requests in 10.10s, 18.35MB read
Requests/sec:  13610.69
Transfer/sec:      1.82MB

$ wrk http://localhost:8000/async
Running 10s test @ http://localhost:8000/async
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   717.14us   49.05us   1.83ms   71.11%
    Req/Sec     7.00k   112.14     7.36k    76.24%
  140613 requests in 10.10s, 18.77MB read
Requests/sec:  13922.97
Transfer/sec:      1.86MB
````

After
```
$ wrk http://localhost:8000/sync
Running 10s test @ http://localhost:8000/sync
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   721.34us  202.40us  11.13ms   99.32%
    Req/Sec     7.01k   230.04     7.62k    94.00%
  139558 requests in 10.00s, 18.63MB read
Requests/sec:  13956.14
Transfer/sec:      1.86MB

$ wrk http://localhost:8000/async
Running 10s test @ http://localhost:8000/async
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   706.04us  109.90us   7.46ms   98.30%
    Req/Sec     7.12k   136.09     7.39k    90.59%
  143188 requests in 10.10s, 19.12MB read
Requests/sec:  14176.95
Transfer/sec:      1.89MB
```

The app used for the test is as follows
```python
from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route
import uvicorn

async def async_page(request):
    return JSONResponse({'status': 'ok'})

async def sync_page(request):
    return JSONResponse({'status': 'ok'})

app = Starlette(routes=[
    Route('/async', async_page),
    Route('/sync', sync_page),
])

if __name__ == "__main__":
    uvicorn.run("app:app", port=8000, log_level="critical")
```

* Apply PR suggestion

---------

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>
  • Loading branch information
sk- and Kludex committed Apr 20, 2024
1 parent 9cf26ee commit 96c90f2
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions starlette/routing.py
Expand Up @@ -63,15 +63,15 @@ def request_response(
Takes a function or coroutine `func(request) -> response`,
and returns an ASGI application.
"""
f: typing.Callable[[Request], typing.Awaitable[Response]] = (
func if is_async_callable(func) else functools.partial(run_in_threadpool, func) # type:ignore
)

async def app(scope: Scope, receive: Receive, send: Send) -> None:
request = Request(scope, receive, send)

async def app(scope: Scope, receive: Receive, send: Send) -> None:
if is_async_callable(func):
response = await func(request)
else:
response = await run_in_threadpool(func, request)
response = await f(request)
await response(scope, receive, send)

await wrap_app_handling_exceptions(app, request)(scope, receive, send)
Expand Down

0 comments on commit 96c90f2

Please sign in to comment.