Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial benchmarks #1243

Merged
merged 8 commits into from
Aug 11, 2022
Merged

Add initial benchmarks #1243

merged 8 commits into from
Aug 11, 2022

Conversation

davidpdrsn
Copy link
Member

This adds a basic setup for writing benchmarks and adds a couple of benchmarks.

The output looks like this

❯ cargo bench -p axum -- send-json
   Compiling axum v0.5.10
    Finished bench [optimized] target(s) in 0.68s
     Running unittests src/lib.rs (target/release/deps/axum-732f709dd139ce93)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 203 filtered out; finished in 0.00s

     Running benches/benches.rs (target/release/deps/benches-ddcccec176823e7f)
Running "send-json" benchmark
  Beginning round 1...
  Benchmarking 10 connections @ http://0.0.0.0:58714 for 10 second(s)
    Latencies:
      Avg      Stdev    Min      Max
      0.06ms   0.03ms   0.02ms   4.06ms
    Requests:
      Total: 1689726 Req/Sec: 168960.39
    Transfer:
      Total: 228.83 MB Transfer Rate: 22.88 MB/Sec

It basically runs a default hyper server and shells out to rewrk which does all the heavy lifting.

I went with rewrk rather than the default cargo bench tooling (or criterion) because I'm mainly interested in seeing requests per second and not how long a single request takes. I would also like to avoid configuring and running our own client. This seems fine to be me but let me know what you think.

Fixes #1199

axum/benches/benches.rs Outdated Show resolved Hide resolved
axum/benches/benches.rs Outdated Show resolved Hide resolved
axum/benches/benches.rs Outdated Show resolved Hide resolved
@jplatte
Copy link
Member

jplatte commented Aug 10, 2022

Overall looks very decent :)

@programatik29
Copy link
Contributor

I observed rewrk doesn't exit with error code on error.
Errors can be detected by checking stderr.

@davidpdrsn
Copy link
Member Author

davidpdrsn commented Aug 11, 2022

I observed rewrk doesn't exit with error code on error.

Errors can be detected by checking stderr.

We should fix that in rewrk. Checking stderr is tedious 😅

@davidpdrsn davidpdrsn enabled auto-merge (squash) August 11, 2022 08:48
@davidpdrsn davidpdrsn merged commit 9e50bf1 into main Aug 11, 2022
@davidpdrsn davidpdrsn deleted the benchmarks branch August 11, 2022 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add benchmarks
3 participants