Skip to content

Commit

Permalink
bench: add new benchmark baseline
Browse files Browse the repository at this point in the history 
I added this so that I can compare the results of the old benchmark
suite with the new one I'm working on in regex-automata. (The idea is to
port all or most of the benchmarks from the old suite and make sure the
results are at least roughly consistent.)
  • Loading branch information
@BurntSushi
BurntSushi committed Jul 1, 2022
1 parent ea525cd commit c01b633
Show file tree
Hide file tree
Showing 4 changed files with 435 additions and 0 deletions.
98 changes: 98 additions & 0 deletions bench/log/09-new-baseline/pcre2
View file
@@ -0,0 +1,98 @@

running 93 tests
test misc::anchored_literal_long_match ... bench: 8 ns/iter (+/- 0) = 48750 MB/s
test misc::anchored_literal_long_non_match ... bench: 7 ns/iter (+/- 0) = 55714 MB/s
test misc::anchored_literal_short_match ... bench: 8 ns/iter (+/- 0) = 3250 MB/s
test misc::anchored_literal_short_non_match ... bench: 7 ns/iter (+/- 0) = 3714 MB/s
test misc::easy0_1K ... bench: 32 ns/iter (+/- 1) = 32843 MB/s
test misc::easy0_1MB ... bench: 22,160 ns/iter (+/- 3,887) = 47319 MB/s
test misc::easy0_32 ... bench: 10 ns/iter (+/- 0) = 5900 MB/s
test misc::easy0_32K ... bench: 651 ns/iter (+/- 2) = 50376 MB/s
test misc::easy1_1K ... bench: 36 ns/iter (+/- 1) = 29000 MB/s
test misc::easy1_1MB ... bench: 22,982 ns/iter (+/- 2,839) = 45626 MB/s
test misc::easy1_32 ... bench: 12 ns/iter (+/- 0) = 4333 MB/s
test misc::easy1_32K ... bench: 654 ns/iter (+/- 2) = 50134 MB/s
test misc::hard_1K ... bench: 469 ns/iter (+/- 9) = 2240 MB/s
test misc::hard_1MB ... bench: 733,962 ns/iter (+/- 28,297) = 1428 MB/s
test misc::hard_32 ... bench: 34 ns/iter (+/- 4) = 1735 MB/s
test misc::hard_32K ... bench: 19,567 ns/iter (+/- 363) = 1676 MB/s
test misc::literal ... bench: 8 ns/iter (+/- 0) = 6375 MB/s
test misc::long_needle1 ... bench: 257,858 ns/iter (+/- 646) = 387 MB/s
test misc::long_needle2 ... bench: 259,045 ns/iter (+/- 2,220) = 386 MB/s
test misc::match_class ... bench: 34 ns/iter (+/- 1) = 2382 MB/s
test misc::match_class_in_range ... bench: 9 ns/iter (+/- 0) = 9000 MB/s
test misc::match_class_unicode ... bench: 125 ns/iter (+/- 3) = 1288 MB/s
test misc::medium_1K ... bench: 35 ns/iter (+/- 3) = 30057 MB/s
test misc::medium_1MB ... bench: 21,126 ns/iter (+/- 4,036) = 49635 MB/s
test misc::medium_32 ... bench: 10 ns/iter (+/- 0) = 6000 MB/s
test misc::medium_32K ... bench: 714 ns/iter (+/- 122) = 45932 MB/s
test misc::not_literal ... bench: 62 ns/iter (+/- 2) = 822 MB/s
test misc::one_pass_long_prefix ... bench: 8 ns/iter (+/- 0) = 3250 MB/s
test misc::one_pass_long_prefix_not ... bench: 8 ns/iter (+/- 0) = 3250 MB/s
test misc::one_pass_short ... bench: 19 ns/iter (+/- 1) = 894 MB/s
test misc::one_pass_short_not ... bench: 19 ns/iter (+/- 1) = 894 MB/s
test misc::reallyhard2_1K ... bench: 1,704 ns/iter (+/- 17) = 610 MB/s
test misc::reallyhard_1K ... bench: 495 ns/iter (+/- 9) = 2123 MB/s
test misc::reallyhard_1MB ... bench: 682,371 ns/iter (+/- 31,284) = 1536 MB/s
test misc::reallyhard_32 ... bench: 34 ns/iter (+/- 2) = 1735 MB/s
test misc::reallyhard_32K ... bench: 17,994 ns/iter (+/- 540) = 1822 MB/s
test misc::reverse_suffix_no_quadratic ... bench: 1,507 ns/iter (+/- 10) = 5308 MB/s
test regexdna::find_new_lines ... bench: 849,983 ns/iter (+/- 25,898) = 5980 MB/s
test regexdna::subst1 ... bench: 520,602 ns/iter (+/- 12,170) = 9764 MB/s
test regexdna::subst10 ... bench: 517,151 ns/iter (+/- 19,060) = 9829 MB/s
test regexdna::subst11 ... bench: 519,209 ns/iter (+/- 12,477) = 9790 MB/s
test regexdna::subst2 ... bench: 513,418 ns/iter (+/- 19,803) = 9901 MB/s
test regexdna::subst3 ... bench: 514,166 ns/iter (+/- 13,019) = 9886 MB/s
test regexdna::subst4 ... bench: 517,808 ns/iter (+/- 30,655) = 9817 MB/s
test regexdna::subst5 ... bench: 516,922 ns/iter (+/- 17,204) = 9834 MB/s
test regexdna::subst6 ... bench: 509,430 ns/iter (+/- 20,608) = 9978 MB/s
test regexdna::subst7 ... bench: 519,437 ns/iter (+/- 10,537) = 9786 MB/s
test regexdna::subst8 ... bench: 520,282 ns/iter (+/- 25,742) = 9770 MB/s
test regexdna::subst9 ... bench: 512,819 ns/iter (+/- 11,443) = 9912 MB/s
test regexdna::variant1 ... bench: 5,302,526 ns/iter (+/- 158,370) = 958 MB/s
test regexdna::variant2 ... bench: 7,421,107 ns/iter (+/- 105,716) = 684 MB/s
test regexdna::variant3 ... bench: 7,310,968 ns/iter (+/- 103,989) = 695 MB/s
test regexdna::variant4 ... bench: 6,152,891 ns/iter (+/- 144,194) = 826 MB/s
test regexdna::variant5 ... bench: 5,717,515 ns/iter (+/- 42,902) = 889 MB/s
test regexdna::variant6 ... bench: 5,840,938 ns/iter (+/- 47,730) = 870 MB/s
test regexdna::variant7 ... bench: 6,624,859 ns/iter (+/- 37,376) = 767 MB/s
test regexdna::variant8 ... bench: 7,308,342 ns/iter (+/- 58,395) = 695 MB/s
test regexdna::variant9 ... bench: 7,372,260 ns/iter (+/- 76,966) = 689 MB/s
test sherlock::before_after_holmes ... bench: 2,817,108 ns/iter (+/- 18,002) = 211 MB/s
test sherlock::before_holmes ... bench: 2,841,515 ns/iter (+/- 14,677) = 209 MB/s
test sherlock::holmes_cochar_watson ... bench: 33,066 ns/iter (+/- 1,766) = 17992 MB/s
test sherlock::ing_suffix ... bench: 1,299,382 ns/iter (+/- 19,674) = 457 MB/s
test sherlock::ing_suffix_limited_space ... bench: 2,799,189 ns/iter (+/- 33,841) = 212 MB/s
test sherlock::letters ... bench: 4,923,399 ns/iter (+/- 111,904) = 120 MB/s
test sherlock::letters_lower ... bench: 5,057,224 ns/iter (+/- 102,860) = 117 MB/s
test sherlock::letters_upper ... bench: 874,306 ns/iter (+/- 10,587) = 680 MB/s
test sherlock::line_boundary_sherlock_holmes ... bench: 15,876 ns/iter (+/- 58) = 37473 MB/s
test sherlock::name_alt1 ... bench: 19,349 ns/iter (+/- 201) = 30747 MB/s
test sherlock::name_alt2 ... bench: 29,916 ns/iter (+/- 581) = 19886 MB/s
test sherlock::name_alt3 ... bench: 461,887 ns/iter (+/- 5,337) = 1288 MB/s
test sherlock::name_alt3_nocase ... bench: 1,813,574 ns/iter (+/- 27,519) = 328 MB/s
test sherlock::name_alt4 ... bench: 30,155 ns/iter (+/- 1,407) = 19729 MB/s
test sherlock::name_alt4_nocase ... bench: 822,605 ns/iter (+/- 56,624) = 723 MB/s
test sherlock::name_alt5 ... bench: 426,318 ns/iter (+/- 12,233) = 1395 MB/s
test sherlock::name_alt5_nocase ... bench: 1,012,097 ns/iter (+/- 27,806) = 587 MB/s
test sherlock::name_holmes ... bench: 19,833 ns/iter (+/- 499) = 29997 MB/s
test sherlock::name_holmes_nocase ... bench: 40,266 ns/iter (+/- 2,089) = 14775 MB/s
test sherlock::name_sherlock ... bench: 14,589 ns/iter (+/- 115) = 40779 MB/s
test sherlock::name_sherlock_holmes ... bench: 14,213 ns/iter (+/- 81) = 41858 MB/s
test sherlock::name_sherlock_holmes_nocase ... bench: 602,296 ns/iter (+/- 98,066) = 987 MB/s
test sherlock::name_sherlock_nocase ... bench: 479,745 ns/iter (+/- 18,070) = 1240 MB/s
test sherlock::name_whitespace ... bench: 14,584 ns/iter (+/- 44) = 40793 MB/s
test sherlock::no_match_common ... bench: 13,499 ns/iter (+/- 1,090) = 44072 MB/s
test sherlock::no_match_really_common ... bench: 12,507 ns/iter (+/- 1,238) = 47568 MB/s
test sherlock::no_match_uncommon ... bench: 11,534 ns/iter (+/- 9) = 51580 MB/s
test sherlock::quotes ... bench: 251,867 ns/iter (+/- 11,818) = 2362 MB/s
test sherlock::repeated_class_negation ... bench: 2,969,330 ns/iter (+/- 287,150) = 200 MB/s
test sherlock::the_lower ... bench: 206,513 ns/iter (+/- 3,294) = 2880 MB/s
test sherlock::the_nocase ... bench: 237,655 ns/iter (+/- 6,616) = 2503 MB/s
test sherlock::the_upper ... bench: 23,922 ns/iter (+/- 510) = 24869 MB/s
test sherlock::the_whitespace ... bench: 326,257 ns/iter (+/- 10,038) = 1823 MB/s
test sherlock::word_ending_n ... bench: 3,264,085 ns/iter (+/- 57,242) = 182 MB/s
test sherlock::words ... bench: 3,161,731 ns/iter (+/- 45,794) = 188 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 93 measured; 0 filtered out; finished in 184.16s

101 changes: 101 additions & 0 deletions bench/log/09-new-baseline/re2
View file
@@ -0,0 +1,101 @@

running 96 tests
test misc::anchored_literal_long_match ... bench: 26 ns/iter (+/- 0) = 15000 MB/s
test misc::anchored_literal_long_non_match ... bench: 4 ns/iter (+/- 0) = 97500 MB/s
test misc::anchored_literal_short_match ... bench: 26 ns/iter (+/- 0) = 1000 MB/s
test misc::anchored_literal_short_non_match ... bench: 4 ns/iter (+/- 0) = 6500 MB/s
test misc::easy0_1K ... bench: 50 ns/iter (+/- 0) = 21020 MB/s
test misc::easy0_1MB ... bench: 51 ns/iter (+/- 0) = 20560843 MB/s
test misc::easy0_32 ... bench: 50 ns/iter (+/- 0) = 1180 MB/s
test misc::easy0_32K ... bench: 50 ns/iter (+/- 0) = 655900 MB/s
test misc::easy1_1K ... bench: 43 ns/iter (+/- 1) = 24279 MB/s
test misc::easy1_1MB ... bench: 43 ns/iter (+/- 0) = 24385953 MB/s
test misc::easy1_32 ... bench: 43 ns/iter (+/- 1) = 1209 MB/s
test misc::easy1_32K ... bench: 43 ns/iter (+/- 0) = 762511 MB/s
test misc::hard_1K ... bench: 50 ns/iter (+/- 0) = 21020 MB/s
test misc::hard_1MB ... bench: 50 ns/iter (+/- 0) = 20972060 MB/s
test misc::hard_32 ... bench: 50 ns/iter (+/- 0) = 1180 MB/s
test misc::hard_32K ... bench: 50 ns/iter (+/- 0) = 655900 MB/s
test misc::literal ... bench: 25 ns/iter (+/- 0) = 2040 MB/s
test misc::long_needle1 ... bench: 356,319 ns/iter (+/- 680) = 280 MB/s
test misc::long_needle2 ... bench: 356,384 ns/iter (+/- 3,126) = 280 MB/s
test misc::match_class ... bench: 94 ns/iter (+/- 0) = 861 MB/s
test misc::match_class_in_range ... bench: 94 ns/iter (+/- 0) = 861 MB/s
test misc::match_class_unicode ... bench: 168 ns/iter (+/- 1) = 958 MB/s
test misc::medium_1K ... bench: 51 ns/iter (+/- 0) = 20627 MB/s
test misc::medium_1MB ... bench: 51 ns/iter (+/- 0) = 20560862 MB/s
test misc::medium_32 ... bench: 51 ns/iter (+/- 0) = 1176 MB/s
test misc::medium_32K ... bench: 51 ns/iter (+/- 1) = 643058 MB/s
test misc::no_exponential ... bench: 112 ns/iter (+/- 0) = 892 MB/s
test misc::not_literal ... bench: 66 ns/iter (+/- 0) = 772 MB/s
test misc::one_pass_long_prefix ... bench: 25 ns/iter (+/- 0) = 1040 MB/s
test misc::one_pass_long_prefix_not ... bench: 44 ns/iter (+/- 0) = 590 MB/s
test misc::one_pass_short ... bench: 43 ns/iter (+/- 0) = 395 MB/s
test misc::one_pass_short_not ... bench: 41 ns/iter (+/- 0) = 414 MB/s
test misc::reallyhard2_1K ... bench: 978 ns/iter (+/- 7) = 1063 MB/s
test misc::reallyhard_1K ... bench: 987 ns/iter (+/- 11) = 1064 MB/s
test misc::reallyhard_1MB ... bench: 957,501 ns/iter (+/- 8,247) = 1095 MB/s
test misc::reallyhard_32 ... bench: 73 ns/iter (+/- 0) = 808 MB/s
test misc::reallyhard_32K ... bench: 30,057 ns/iter (+/- 315) = 1091 MB/s
test misc::reverse_suffix_no_quadratic ... bench: 7,368 ns/iter (+/- 105) = 1085 MB/s
test regexdna::find_new_lines ... bench: 15,567,882 ns/iter (+/- 48,213) = 326 MB/s
test regexdna::subst1 ... bench: 2,011,288 ns/iter (+/- 23,092) = 2527 MB/s
test regexdna::subst10 ... bench: 2,013,337 ns/iter (+/- 33,388) = 2524 MB/s
test regexdna::subst11 ... bench: 2,005,968 ns/iter (+/- 25,799) = 2534 MB/s
test regexdna::subst2 ... bench: 2,022,572 ns/iter (+/- 23,311) = 2513 MB/s
test regexdna::subst3 ... bench: 2,018,386 ns/iter (+/- 32,071) = 2518 MB/s
test regexdna::subst4 ... bench: 2,013,345 ns/iter (+/- 32,599) = 2524 MB/s
test regexdna::subst5 ... bench: 2,015,871 ns/iter (+/- 25,081) = 2521 MB/s
test regexdna::subst6 ... bench: 2,008,492 ns/iter (+/- 24,502) = 2530 MB/s
test regexdna::subst7 ... bench: 2,018,804 ns/iter (+/- 38,700) = 2518 MB/s
test regexdna::subst8 ... bench: 2,010,856 ns/iter (+/- 23,695) = 2527 MB/s
test regexdna::subst9 ... bench: 2,023,767 ns/iter (+/- 17,040) = 2511 MB/s
test regexdna::variant1 ... bench: 4,688,839 ns/iter (+/- 19,258) = 1084 MB/s
test regexdna::variant2 ... bench: 4,693,463 ns/iter (+/- 31,741) = 1083 MB/s
test regexdna::variant3 ... bench: 4,674,020 ns/iter (+/- 15,755) = 1087 MB/s
test regexdna::variant4 ... bench: 4,666,017 ns/iter (+/- 16,318) = 1089 MB/s
test regexdna::variant5 ... bench: 4,682,965 ns/iter (+/- 17,552) = 1085 MB/s
test regexdna::variant6 ... bench: 4,661,825 ns/iter (+/- 21,667) = 1090 MB/s
test regexdna::variant7 ... bench: 4,697,959 ns/iter (+/- 24,282) = 1082 MB/s
test regexdna::variant8 ... bench: 4,700,703 ns/iter (+/- 21,377) = 1081 MB/s
test regexdna::variant9 ... bench: 4,665,298 ns/iter (+/- 19,086) = 1089 MB/s
test sherlock::before_after_holmes ... bench: 560,350 ns/iter (+/- 3,852) = 1061 MB/s
test sherlock::before_holmes ... bench: 574,423 ns/iter (+/- 4,638) = 1035 MB/s
test sherlock::everything_greedy ... bench: 2,688,852 ns/iter (+/- 16,320) = 221 MB/s
test sherlock::everything_greedy_nl ... bench: 1,206,136 ns/iter (+/- 6,173) = 493 MB/s
test sherlock::holmes_cochar_watson ... bench: 547,910 ns/iter (+/- 7,147) = 1085 MB/s
test sherlock::holmes_coword_watson ... bench: 610,803 ns/iter (+/- 1,029) = 974 MB/s
test sherlock::ing_suffix ... bench: 777,478 ns/iter (+/- 3,028) = 765 MB/s
test sherlock::ing_suffix_limited_space ... bench: 725,653 ns/iter (+/- 4,746) = 819 MB/s
test sherlock::letters ... bench: 25,265,004 ns/iter (+/- 120,234) = 23 MB/s
test sherlock::letters_lower ... bench: 24,615,621 ns/iter (+/- 134,875) = 24 MB/s
test sherlock::letters_upper ... bench: 1,485,920 ns/iter (+/- 21,446) = 400 MB/s
test sherlock::line_boundary_sherlock_holmes ... bench: 547,229 ns/iter (+/- 3,726) = 1087 MB/s
test sherlock::name_alt1 ... bench: 18,148 ns/iter (+/- 131) = 32782 MB/s
test sherlock::name_alt2 ... bench: 586,335 ns/iter (+/- 3,679) = 1014 MB/s
test sherlock::name_alt3 ... bench: 601,096 ns/iter (+/- 3,781) = 989 MB/s
test sherlock::name_alt3_nocase ... bench: 602,319 ns/iter (+/- 7,872) = 987 MB/s
test sherlock::name_alt4 ... bench: 586,762 ns/iter (+/- 3,465) = 1013 MB/s
test sherlock::name_alt4_nocase ... bench: 595,539 ns/iter (+/- 3,240) = 998 MB/s
test sherlock::name_alt5 ... bench: 592,474 ns/iter (+/- 6,361) = 1004 MB/s
test sherlock::name_alt5_nocase ... bench: 593,214 ns/iter (+/- 4,667) = 1002 MB/s
test sherlock::name_holmes ... bench: 40,236 ns/iter (+/- 514) = 14786 MB/s
test sherlock::name_holmes_nocase ... bench: 215,216 ns/iter (+/- 4,822) = 2764 MB/s
test sherlock::name_sherlock ... bench: 14,064 ns/iter (+/- 159) = 42301 MB/s
test sherlock::name_sherlock_holmes ... bench: 15,727 ns/iter (+/- 166) = 37828 MB/s
test sherlock::name_sherlock_holmes_nocase ... bench: 552,042 ns/iter (+/- 6,395) = 1077 MB/s
test sherlock::name_sherlock_nocase ... bench: 552,475 ns/iter (+/- 5,365) = 1076 MB/s
test sherlock::name_whitespace ... bench: 16,210 ns/iter (+/- 194) = 36701 MB/s
test sherlock::no_match_common ... bench: 147,489 ns/iter (+/- 602) = 4033 MB/s
test sherlock::no_match_really_common ... bench: 157,205 ns/iter (+/- 350) = 3784 MB/s
test sherlock::no_match_uncommon ... bench: 4,849 ns/iter (+/- 5) = 122691 MB/s
test sherlock::quotes ... bench: 619,880 ns/iter (+/- 5,189) = 959 MB/s
test sherlock::the_lower ... bench: 685,396 ns/iter (+/- 12,559) = 868 MB/s
test sherlock::the_nocase ... bench: 771,051 ns/iter (+/- 18,470) = 771 MB/s
test sherlock::the_upper ... bench: 59,139 ns/iter (+/- 1,604) = 10059 MB/s
test sherlock::the_whitespace ... bench: 736,147 ns/iter (+/- 7,668) = 808 MB/s
test sherlock::word_ending_n ... bench: 1,200,401 ns/iter (+/- 11,206) = 495 MB/s
test sherlock::words ... bench: 8,024,768 ns/iter (+/- 93,051) = 74 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 96 measured; 0 filtered out; finished in 86.80s

0 comments on commit c01b633

Please sign in to comment.