s2: Add compression estimate #752
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add `EstimateBlockSize` that ``` BenchmarkEncodeS2BlockParallel/0-html/est-size-16 207572 5746 ns/op 17822.54 MB/s 22123 B 21.60 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/0-html/block-16 190375 6058 ns/op 16904.40 MB/s 20868 B 20.38 pct 19 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/0-html/block-better-16 89342 13496 ns/op 7587.43 MB/s 18972 B 18.53 pct 40 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/0-html/block-best-16 5635 204202 ns/op 501.46 MB/s 17403 B 17.00 pct 643 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/est-size-16 18522 63202 ns/op 11108.55 MB/s 313575 B 44.66 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/block-16 16382 72096 ns/op 9738.19 MB/s 286541 B 40.81 pct 1462 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/block-better-16 8055 148821 ns/op 4717.66 MB/s 248079 B 35.33 pct 2974 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/block-best-16 446 2436297 ns/op 288.18 MB/s 229314 B 32.66 pct 53724 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/est-size-16 10415599 113.6 ns/op 1083574.52 MB/s 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/block-16 2596231 456.3 ns/op 269737.56 MB/s 123100 B 100.0 pct 1 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/block-better-16 704770 1561 ns/op 78870.09 MB/s 123100 B 100.0 pct 6 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/block-best-16 14368 74375 ns/op 1655.04 MB/s 123025 B 99.94 pct 310 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/est-size-16 99533023 12.35 ns/op 16198.92 MB/s 151.0 B 75.50 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/block-16 107843694 11.02 ns/op 18151.08 MB/s 155.0 B 77.50 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/block-better-16 39622100 29.80 ns/op 6710.82 MB/s 149.0 B 74.50 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/block-best-16 46939 24423 ns/op 8.19 MB/s 142.0 B 71.00 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/est-size-16 1563739 761.8 ns/op 134418.16 MB/s 84867 B 82.88 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/block-16 1000000 1013 ns/op 101124.34 MB/s 84202 B 82.23 pct 3 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/block-better-16 114718 10198 ns/op 10041.40 MB/s 82887 B 80.94 pct 31 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/block-best-16 3022 397183 ns/op 257.82 MB/s 82327 B 80.40 pct 1198 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/est-size-16 167383 7194 ns/op 56935.67 MB/s 46130 B 11.26 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/block-16 137107 8084 ns/op 50670.44 MB/s 20870 B 5.095 pct 102 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/block-better-16 80835 14473 ns/op 28301.20 MB/s 18982 B 4.634 pct 174 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/block-best-16 5659 214283 ns/op 1911.49 MB/s 17411 B 4.251 pct 2487 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/est-size-16 54757 21891 ns/op 6947.42 MB/s 90661 B 59.61 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/block-16 49647 24292 ns/op 6260.82 MB/s 85934 B 56.50 pct 106 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/block-better-16 27152 44545 ns/op 3414.30 MB/s 71611 B 47.08 pct 194 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/block-best-16 1551 760707 ns/op 199.93 MB/s 66182 B 43.52 pct 3413 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/est-size-16 69514 17415 ns/op 7188.19 MB/s 83328 B 66.57 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/block-16 62397 19237 ns/op 6507.18 MB/s 79575 B 63.57 pct 71 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/block-better-16 30704 38667 ns/op 3237.35 MB/s 65941 B 52.68 pct 145 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/block-best-16 1665 680136 ns/op 184.05 MB/s 61870 B 49.43 pct 2677 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/est-size-16 19372 61609 ns/op 6926.83 MB/s 233737 B 54.77 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/block-16 16821 69995 ns/op 6096.93 MB/s 220383 B 51.64 pct 877 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/block-better-16 9943 121114 ns/op 3523.56 MB/s 184939 B 43.34 pct 1484 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/block-best-16 550 1956728 ns/op 218.10 MB/s 167926 B 39.35 pct 26843 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/est-size-16 17184 70208 ns/op 6863.34 MB/s 341582 B 70.89 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/block-16 14602 81018 ns/op 5947.60 MB/s 318196 B 66.03 pct 1125 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/block-better-16 7519 160529 ns/op 3001.70 MB/s 264990 B 54.99 pct 2185 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/block-best-16 351 2914298 ns/op 165.34 MB/s 242003 B 50.22 pct 46822 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/est-size-16 267162 4371 ns/op 27130.39 MB/s 21121 B 17.81 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/block-16 255237 4636 ns/op 25579.35 MB/s 18606 B 15.69 pct 16 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/block-better-16 97609 11808 ns/op 10042.76 MB/s 17689 B 14.92 pct 42 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/block-best-16 5970 189710 ns/op 625.10 MB/s 16011 B 13.50 pct 700 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/est-size-16 56253 21214 ns/op 8688.63 MB/s 67091 B 36.40 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/block-16 55222 21493 ns/op 8575.72 MB/s 65019 B 35.28 pct 116 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/block-better-16 32078 37589 ns/op 4903.59 MB/s 55398 B 30.06 pct 199 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/block-best-16 2006 576210 ns/op 319.88 MB/s 49728 B 26.98 pct 3194 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/est-size-16 141980487 8.482 ns/op 15090.24 MB/s 82.00 B 64.06 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/block-16 148818110 8.218 ns/op 15575.59 MB/s 82.00 B 64.06 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/block-better-16 62333452 18.34 ns/op 6979.91 MB/s 78.00 B 60.94 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/block-best-16 46005 28094 ns/op 4.56 MB/s 78.00 B 60.94 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/est-size-16 17325642 70.00 ns/op 14286.61 MB/s 794.0 B 79.40 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/block-16 14739968 79.66 ns/op 12553.29 MB/s 774.0 B 77.40 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/block-better-16 6493140 182.6 ns/op 5475.71 MB/s 746.0 B 74.60 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/block-best-16 28593 42199 ns/op 23.70 MB/s 742.0 B 74.20 pct 1 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/est-size-16 1650962 731.2 ns/op 13675.38 MB/s 7357 B 73.57 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/block-16 1504424 794.1 ns/op 12592.21 MB/s 6933 B 69.33 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/block-better-16 683931 1731 ns/op 5778.64 MB/s 6218 B 62.18 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/block-best-16 8996 127511 ns/op 78.42 MB/s 6018 B 60.18 pct 38 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/est-size-16 746936 1475 ns/op 13562.68 MB/s 13736 B 68.68 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/block-16 683935 1699 ns/op 11770.06 MB/s 13516 B 67.58 pct 1 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/block-better-16 199888 6283 ns/op 3183.04 MB/s 11492 B 57.46 pct 3 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/block-best-16 8083 148602 ns/op 134.59 MB/s 11013 B 55.06 pct 86 B/op 0 allocs/op ``` Noasm: ``` goos: windows goarch: amd64 pkg: github.com/klauspost/compress/s2 cpu: AMD Ryzen 9 3950X 16-Core Processor BenchmarkEncodeS2BlockParallel/0-html/est-size-16 206064 5611 ns/op 18251.05 MB/s 22123 B 21.60 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/0-html/block-16 193548 6043 ns/op 16944.16 MB/s 20868 B 20.38 pct 18 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/0-html/block-better-16 92278 13271 ns/op 7716.26 MB/s 18972 B 18.53 pct 39 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/0-html/block-best-16 6012 211476 ns/op 484.22 MB/s 17403 B 17.00 pct 602 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/est-size-16 18085 64505 ns/op 10884.25 MB/s 313575 B 44.66 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/block-16 16641 72254 ns/op 9716.94 MB/s 286541 B 40.81 pct 1440 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/block-better-16 8190 147021 ns/op 4775.42 MB/s 248079 B 35.33 pct 2925 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/1-urls/block-best-16 457 2346591 ns/op 299.19 MB/s 229314 B 32.66 pct 52431 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/est-size-16 9996766 114.1 ns/op 1078424.08 MB/s 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/block-16 2554573 481.3 ns/op 255727.67 MB/s 123100 B 100.0 pct 1 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/block-better-16 725973 1620 ns/op 75983.90 MB/s 123100 B 100.0 pct 6 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/2-jpg/block-best-16 15980 86360 ns/op 1425.35 MB/s 123025 B 99.94 pct 279 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/est-size-16 97503404 12.30 ns/op 16262.94 MB/s 151.0 B 75.50 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/block-16 100000000 11.17 ns/op 17906.04 MB/s 155.0 B 77.50 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/block-better-16 42084589 29.70 ns/op 6733.93 MB/s 149.0 B 74.50 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/3-jpg_200b/block-best-16 46666 30233 ns/op 6.62 MB/s 142.0 B 71.00 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/est-size-16 1603496 767.1 ns/op 133496.35 MB/s 84867 B 82.88 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/block-16 1209092 976.1 ns/op 104904.83 MB/s 84202 B 82.23 pct 2 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/block-better-16 120744 9637 ns/op 10626.04 MB/s 82887 B 80.94 pct 30 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/4-pdf/block-best-16 3207 360495 ns/op 284.05 MB/s 82327 B 80.40 pct 1129 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/est-size-16 171427 6953 ns/op 58913.45 MB/s 46130 B 11.26 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/block-16 147021 7847 ns/op 52195.52 MB/s 20870 B 5.095 pct 95 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/block-better-16 77374 14038 ns/op 29178.20 MB/s 18982 B 4.634 pct 181 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/5-html4/block-best-16 6410 195077 ns/op 2099.68 MB/s 17411 B 4.251 pct 2195 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/est-size-16 56328 21296 ns/op 7141.57 MB/s 90661 B 59.61 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/block-16 49772 23576 ns/op 6451.05 MB/s 85934 B 56.50 pct 106 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/block-better-16 28126 42782 ns/op 3554.96 MB/s 71611 B 47.08 pct 188 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/6-txt1/block-best-16 1561 695629 ns/op 218.64 MB/s 66182 B 43.52 pct 3391 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/est-size-16 72219 16478 ns/op 7596.93 MB/s 83328 B 66.57 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/block-16 63639 18683 ns/op 6700.25 MB/s 79575 B 63.57 pct 70 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/block-better-16 32029 37419 ns/op 3345.29 MB/s 65941 B 52.68 pct 139 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/7-txt2/block-best-16 1807 637784 ns/op 196.27 MB/s 61870 B 49.43 pct 2467 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/est-size-16 19794 60379 ns/op 7067.93 MB/s 233737 B 54.77 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/block-16 17246 69432 ns/op 6146.33 MB/s 220383 B 51.64 pct 856 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/block-better-16 10083 119233 ns/op 3579.17 MB/s 184939 B 43.34 pct 1464 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/8-txt3/block-best-16 590 1945892 ns/op 219.31 MB/s 167926 B 39.35 pct 25023 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/est-size-16 17264 69558 ns/op 6927.52 MB/s 341582 B 70.89 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/block-16 14724 80751 ns/op 5967.24 MB/s 318196 B 66.03 pct 1116 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/block-better-16 7543 158070 ns/op 3048.41 MB/s 264990 B 54.99 pct 2178 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/9-txt4/block-best-16 384 2815967 ns/op 171.12 MB/s 242003 B 50.22 pct 42799 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/est-size-16 266250 4315 ns/op 27479.71 MB/s 21121 B 17.81 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/block-16 244342 4566 ns/op 25973.35 MB/s 18606 B 15.69 pct 17 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/block-better-16 97837 11494 ns/op 10317.35 MB/s 17689 B 14.92 pct 42 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/10-pb/block-best-16 6297 185490 ns/op 639.32 MB/s 16011 B 13.50 pct 663 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/est-size-16 56702 21138 ns/op 8719.77 MB/s 67091 B 36.40 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/block-16 55410 21386 ns/op 8618.87 MB/s 65019 B 35.28 pct 115 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/block-better-16 31664 36989 ns/op 4983.05 MB/s 55398 B 30.06 pct 202 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/11-gaviota/block-best-16 2055 562326 ns/op 327.78 MB/s 49728 B 26.98 pct 3118 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/est-size-16 142262313 8.389 ns/op 15257.94 MB/s 82.00 B 64.06 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/block-16 149785786 8.201 ns/op 15607.45 MB/s 82.00 B 64.06 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/block-better-16 65669124 18.24 ns/op 7018.39 MB/s 78.00 B 60.94 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/12-txt1_128b/block-best-16 47752 23642 ns/op 5.41 MB/s 78.00 B 60.94 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/est-size-16 16548277 66.20 ns/op 15106.26 MB/s 794.0 B 79.40 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/block-16 15555889 76.06 ns/op 13148.16 MB/s 774.0 B 77.40 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/block-better-16 6623119 183.0 ns/op 5465.18 MB/s 746.0 B 74.60 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/13-txt1_1000b/block-best-16 35971 35347 ns/op 28.29 MB/s 742.0 B 74.20 pct 1 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/est-size-16 1632084 723.1 ns/op 13828.64 MB/s 7357 B 73.57 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/block-16 1413682 786.7 ns/op 12711.68 MB/s 6933 B 69.33 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/block-better-16 682802 1710 ns/op 5848.59 MB/s 6218 B 62.18 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/14-txt1_10000b/block-best-16 9460 115376 ns/op 86.67 MB/s 6018 B 60.18 pct 36 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/est-size-16 750022 1462 ns/op 13677.87 MB/s 13736 B 68.68 pct 0 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/block-16 682224 1696 ns/op 11791.21 MB/s 13516 B 67.58 pct 1 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/block-better-16 190477 6060 ns/op 3300.47 MB/s 11492 B 57.46 pct 3 B/op 0 allocs/op BenchmarkEncodeS2BlockParallel/15-txt1_20000b/block-best-16 8642 148052 ns/op 135.09 MB/s 11013 B 55.06 pct 80 B/op 0 allocs/op ```
# Conflicts: # s2/_generate/gen.go # s2/encode_go.go
klauspost
force-pushed
the
s2-add-compression-estimate
branch
from
83106f9
to
406731b
Compare
kodiakhq bot
pushed a commit
to cloudquery/filetypes
that referenced
this pull request
Mar 1, 2023
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/klauspost/compress](https://togithub.com/klauspost/compress) | indirect | minor | `v1.15.11` -> `v1.16.0` | --- ### ⚠ Dependency Lookup Warnings ⚠ Warnings were logged while processing this repo. Please check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>klauspost/compress</summary> ### [`v1.16.0`](https://togithub.com/klauspost/compress/releases/tag/v1.16.0) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.15.15...v1.16.0) #### What's Changed - s2: Add Dictionary support by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#685 - s2: Add Compression Size Estimate by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#752 - s2: Add support for custom stream encoder by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#755 - s2: Add LZ4 block converter by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#748 - s2: Support io.ReaderAt in ReadSeeker by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#747 - s2c/s2sx: Use concurrent decoding by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#746 - tests: Upgrade to Go 1.20 by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#749 - Update all (command) dependencies by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#758 **Full Changelog**: klauspost/compress@v1.15.15...v1.16.0 ### [`v1.15.15`](https://togithub.com/klauspost/compress/releases/tag/v1.15.15) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.15.14...v1.15.15) #### What's Changed - zstd: Add delta encoding support by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#728 - huff0: Reduce bounds checking by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#734 - huff0: Assembler improvements by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#736 - deflate: Improve level 7-9 by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#739 - gzhttp: Add SuffixETag() and DropETag() options to prevent ETag collisions on compressed responses by [@​willbicks](https://togithub.com/willbicks) in [klauspost/compress#740 - zstd: Don't allocate dataStorage when using byteBuf by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#741 - huff0: Speed up compression of short blocks by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#744 - zstd: Handle dicts by pointer, always by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#743 - fse: Optimize compression by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#745 - Retract v1.14.1-v.1.14.3 by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#742 #### New Contributors - [@​willbicks](https://togithub.com/willbicks) made their first contribution in [klauspost/compress#740 **Full Changelog**: klauspost/compress@v1.15.14...v1.15.15 ### [`v1.15.14`](https://togithub.com/klauspost/compress/releases/tag/v1.15.14) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.15.13...v1.15.14) #### What's Changed - flate: Improve speed in big stateless blocks. by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#718 - zstd: Trigger BCE by switching on lengths by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#716 - zstd: Shave some instructions off the amd64 asm by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#720 - export NoGzipResponseWriter for custom ResponseWriter wrappers by [@​harshavardhana](https://togithub.com/harshavardhana) in [klauspost/compress#722 - s2: Add example for indexing and existing stream by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#723 - tests: Tweak fuzz tests by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#719 #### New Contributors - [@​harshavardhana](https://togithub.com/harshavardhana) made their first contribution in [klauspost/compress#722 **Full Changelog**: klauspost/compress@v1.15.13...v1.15.14 ### [`v1.15.13`](https://togithub.com/klauspost/compress/releases/tag/v1.15.13) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.15.12...v1.15.13) #### What's Changed - zstd: Add MaxEncodedSize to encoder by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#691 - zstd: Improve "best" end search by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#693 - zstd: Replace bytes.Equal with smaller comparisons by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#695 - zstd: Faster CRC checking/skipping by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#696 - zstd: Rewrite matchLen to make it inlineable by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#701 - zstd: Write table clearing in a way that the compiler recognizes by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#702 - zstd: Use individual reset threshold by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#703 - huff0: Check for zeros earlier in Scratch.countSimple by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#704 - zstd: Improve best compression's match selection by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#705 - zstd: Select best match using selection trees by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#706 - zstd: sync xxhash with final accepted patch upstream by [@​lizthegrey](https://togithub.com/lizthegrey) in [klauspost/compress#707 - zstd: Import xxhash v2.2.0 by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#708 **Full Changelog**: klauspost/compress@v1.15.12...v1.15.13 ### [`v1.15.12`](https://togithub.com/klauspost/compress/releases/tag/v1.15.12) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.15.11...v1.15.12) #### What's Changed - zstd: Tweak decoder allocs. by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#680 - gzhttp: Always delete `HeaderNoCompression` by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#683 **Full Changelog**: klauspost/compress@v1.15.11...v1.15.12 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 3am on the first day of the month" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xMDkuMSIsInVwZGF0ZWRJblZlciI6IjM0LjE1NC4wIn0=-->
kodiakhq bot
pushed a commit
to cloudquery/plugin-pb-go
that referenced
this pull request
Aug 1, 2023
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/klauspost/compress](https://togithub.com/klauspost/compress) | indirect | minor | `v1.15.15` -> `v1.16.7` | --- ### Release Notes <details> <summary>klauspost/compress (github.com/klauspost/compress)</summary> ### [`v1.16.7`](https://togithub.com/klauspost/compress/releases/tag/v1.16.7) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.6...v1.16.7) #### What's Changed - zstd: Fix default level first dictionary encode by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#829 - docs: Fix typo in security advisory URL by [@​vcabbage](https://togithub.com/vcabbage) in [klauspost/compress#830 - s2: add GetBufferCapacity() method by [@​GiedriusS](https://togithub.com/GiedriusS) in [klauspost/compress#832 #### New Contributors - [@​vcabbage](https://togithub.com/vcabbage) made their first contribution in [klauspost/compress#830 - [@​GiedriusS](https://togithub.com/GiedriusS) made their first contribution in [klauspost/compress#832 **Full Changelog**: klauspost/compress@v1.16.6...v1.16.7 ### [`v1.16.6`](https://togithub.com/klauspost/compress/releases/tag/v1.16.6) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.5...v1.16.6) #### What's Changed - zstd: correctly ignore WithEncoderPadding(1) by [@​ianlancetaylor](https://togithub.com/ianlancetaylor) in [klauspost/compress#806 - gzhttp: Handle informational headers by [@​rtribotte](https://togithub.com/rtribotte) in [klauspost/compress#815 - zstd: Add amd64 match length assembly by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#824 - s2: Improve Better compression slightly by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#663 - s2: Clean up matchlen assembly by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#825 #### New Contributors - [@​rtribotte](https://togithub.com/rtribotte) made their first contribution in [klauspost/compress#815 - [@​dveeden](https://togithub.com/dveeden) made their first contribution in [klauspost/compress#816 **Full Changelog**: klauspost/compress@v1.16.5...v1.16.6 ### [`v1.16.5`](https://togithub.com/klauspost/compress/releases/tag/v1.16.5) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.4...v1.16.5) #### What's Changed - zstd: readByte needs to use io.ReadFull by [@​jnoxon](https://togithub.com/jnoxon) in [klauspost/compress#802 - gzip: Fix WriterTo after initial read by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#804 #### New Contributors - [@​jnoxon](https://togithub.com/jnoxon) made their first contribution in [klauspost/compress#802 **Full Changelog**: klauspost/compress@v1.16.4...v1.16.5 ### [`v1.16.4`](https://togithub.com/klauspost/compress/releases/tag/v1.16.4) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.3...v1.16.4) #### What's Changed - s2: Fix huge block overflow by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#779 - s2: Allow CustomEncoder fallback by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#780 - zstd: Fix amd64 not always detecting corrupt data by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#785 - zstd: Improve zstd best efficiency by [@​klauspost](https://togithub.com/klauspost) and [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#784 - zstd: Make load(32|64)32 safer and smaller by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#788 - zstd: Fix quick reject on long backmatches by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#787 - zstd: Revert table size change by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#789 - zstd: Respect WithAllLitEntropyCompression by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#792 - zstd: Fix back-referenced offset by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#793 - zstd: Load source value at start of loop by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#794 - zstd: Shorten checksum code by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#795 - zstd: Fix fallback on incompressible block by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#798 - gzhttp: Suppport ResponseWriter Unwrap() in gzhttp handler by [@​jgimenez](https://togithub.com/jgimenez) in [klauspost/compress#799 #### New Contributors - [@​jgimenez](https://togithub.com/jgimenez) made their first contribution in [klauspost/compress#799 **Full Changelog**: klauspost/compress@v1.16.3...v1.16.4 ### [`v1.16.3`](https://togithub.com/klauspost/compress/releases/tag/v1.16.3) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.2...v1.16.3) **Full Changelog**: klauspost/compress@v1.16.2...v1.16.3 ### [`v1.16.2`](https://togithub.com/klauspost/compress/releases/tag/v1.16.2) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.1...v1.16.2) #### What's Changed - Fix Goreleaser permissions by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#777 **Full Changelog**: klauspost/compress@v1.16.1...v1.16.2 ### [`v1.16.1`](https://togithub.com/klauspost/compress/releases/tag/v1.16.1) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.16.0...v1.16.1) #### What's Changed - zstd: Speed up + improve best encoder by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#776 - s2: Add Intel LZ4s converter by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#766 - gzhttp: Add BREACH mitigation by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#762 - gzhttp: Remove a few unneeded allocs by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#768 - gzhttp: Fix crypto/rand.Read usage by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#770 - gzhttp: Use SHA256 as paranoid option by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#769 - gzhttp: Use strings for randomJitter to skip a copy by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#767 - zstd: Fix ineffective block size check by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#771 - zstd: Check FSE init values by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#772 - zstd: Report EOF from byteBuf.readBig by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#773 - huff0: Speed up compress1xDo by [@​greatroar](https://togithub.com/greatroar) in [klauspost/compress#774 - tests: Remove fuzz printing by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#775 - tests: Add CICD Fuzz testing by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#763 - ci: set minimal permissions to GitHub Workflows by [@​diogoteles08](https://togithub.com/diogoteles08) in [klauspost/compress#765 #### New Contributors - [@​diogoteles08](https://togithub.com/diogoteles08) made their first contribution in [klauspost/compress#765 **Full Changelog**: klauspost/compress@v1.16.0...v1.16.1 ### [`v1.16.0`](https://togithub.com/klauspost/compress/releases/tag/v1.16.0) [Compare Source](https://togithub.com/klauspost/compress/compare/v1.15.15...v1.16.0) #### What's Changed - s2: Add Dictionary support by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#685 - s2: Add Compression Size Estimate by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#752 - s2: Add support for custom stream encoder by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#755 - s2: Add LZ4 block converter by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#748 - s2: Support io.ReaderAt in ReadSeeker by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#747 - s2c/s2sx: Use concurrent decoding by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#746 - tests: Upgrade to Go 1.20 by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#749 - Update all (command) dependencies by [@​klauspost](https://togithub.com/klauspost) in [klauspost/compress#758 **Full Changelog**: klauspost/compress@v1.15.15...v1.16.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 4am on the first day of the month" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4yNi4xIiwidXBkYXRlZEluVmVyIjoiMzYuMjYuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add
EstimateBlockSizethat will perform a very fast compressionwithout outputting the result and return the compressed output size.
The function returns -1 if no improvement could be achieved.
Using actual compression will most often produce better compression than the estimate.
Noasm: