Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow comparison against a baseline #28

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

michaelherold
Copy link
Owner

This change introduces a new comparison ordering strategy where the initial report serves as the baseline against which the job compares the other reports.

This is useful in cases where you're testing out different implementations and want to see whether they are better or worse (and the comparative increase or decrease) in an easy-to-decipher order.

Closes #26

This change introduces a new comparison ordering strategy where the
initial report serves as the baseline against which the job compares the
other reports.

This is useful in cases where you're testing out different
implementations and want to see whether they are better or worse (and
the comparative increase or decrease) in an easy-to-decipher order.
@technicalpickles
Copy link

Thanks for putting this together! I tried this with a benchmark I had kicking around: https://gist.github.com/technicalpickles/c59d3e59dfc9f965e802045684fb4ebd

The order looks correct, but the relative numbers don't look quite right:

Comparison:
            original:       1680 allocated - baseline
map! with capitalize! & logical or:        760 allocated - same
map! capitalize! & two statements:        760 allocated - same
    each capitalize!:        760 allocated - same
                map!:        760 allocated - same
        string split:        840 allocated - same
         capitalize!:        840 allocated - same
            gsub alt:       1576 allocated - same
                gsub:       1816 allocated - 1.08x more

It's showing same for everything that has less memory allocation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Order comparison results by baseline similar to benchmark-ips
2 participants