Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add workflow to run benchmarks #642

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Mingun
Copy link
Collaborator

@Mingun Mingun commented Sep 5, 2023

Using https://github.com/boa-dev/criterion-compare-action. Let's see what it can give us

@dralley
Copy link
Collaborator

dralley commented Sep 5, 2023

There's just too much noise with github actions to get good results. It looks like there are swings of up to 13% in the test run.

The criteron book recommends using something like IaI instead https://github.com/bheisler/criterion.rs/blob/master/book/src/faq.md#how-should-i-run-criterionrs-benchmarks-in-a-ci-pipeline

@Mingun
Copy link
Collaborator Author

Mingun commented Sep 5, 2023

Yeah, I read that, but if such actions is exist, it would be interesting to check them. Actually, I want to see a graph somewhere, similar to which https://github.com/benchmark-action/github-action-benchmark is produced, so to have an idea of performance, but that action unfortunately doesn't work with criterion.

I agree, that it is not worth to merge it right now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants