Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks #1199

Closed
davidpdrsn opened this issue Jul 27, 2022 · 0 comments · Fixed by #1243
Closed

Add benchmarks #1199

davidpdrsn opened this issue Jul 27, 2022 · 0 comments · Fixed by #1243
Labels
C-performance Category: Performance related issues and PRs E-help-wanted Call for participation: Help is requested to fix this issue.

Comments

@davidpdrsn
Copy link
Member

davidpdrsn commented Jul 27, 2022

The recent discussions around the TechEmpower benchmarks showed that, while axum is plenty fast, having some benchmarks in the repo would be useful:

  • The purpose of the benchmarks should be to compare axum to itself such that one can easily see if some change improves performance.
  • I don't they should do git shenanigans to automatically compare to a previous commit. Just print numbers for the current commit.
  • They should not be designed to compare axum to other frameworks. There are other benchmarks that do that.
  • They should be easy to run. Ideally just cargo bench if the output is good enough. Some bash script using rewrk is also fine with me.
  • They should exercise things most users will use like routing and JSON input/output. Non axum specifics, like databases, aren't relevant imo.
  • It's important that the benchmarks don't use unrealistic code. They should use the APIs we expect users to use.
  • I don't necessarily think we should run the benchmarks on CI but should at least type check them to prevent bit rot.

I don't intend to chase micro benchmarks but having some convenient scripts to run does have value imo.

@davidpdrsn davidpdrsn added E-help-wanted Call for participation: Help is requested to fix this issue. C-performance Category: Performance related issues and PRs labels Jul 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-performance Category: Performance related issues and PRs E-help-wanted Call for participation: Help is requested to fix this issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant