Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: Add performance regression monitoring for pull requests #1411

Merged
merged 22 commits into from Oct 14, 2021
Merged

CI: Add performance regression monitoring for pull requests #1411

merged 22 commits into from Oct 14, 2021

Conversation

larabr
Copy link
Collaborator

@larabr larabr commented Sep 1, 2021 •
edited

Add tests to monitor performance of the main top-level functions, and run them on pull requests:

  • if performance is 30% worse than the master branch, an alert comment will be left on the PR commit. A lower threshold risks giving false positive regression alerts: during testing, I've observed a variability of up to 20% when testing the same code.
  • if performance is 50% worse, the workflow will fail.

For example workflow behaviour and the definitive alert comments, see larabr#2.

@larabr larabr requested a review from twiss September 1, 2021 17:14
@larabr larabr mentioned this pull request Sep 9, 2021
Merged
2 tasks
twiss
twiss reviewed Oct 7, 2021
View reviewed changes
.github/workflows/benchmark.yml Outdated Show resolved Hide resolved
test/benchmarks/benchmark.js Show resolved Hide resolved
@larabr larabr requested a review from twiss October 8, 2021 17:04
@larabr larabr merged commit 421733e into openpgpjs:master Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twiss twiss twiss approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@larabr @twiss