Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible alternative to our latin1_to_utf8_avx512 routine. #346

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

lemire
Copy link
Member

@lemire lemire commented Nov 3, 2023

We only change the bulk of the processing, leaving the tail intact. It is currently no faster.

GCC 12, Icelake.

Main branch:

convert_latin1_to_utf8+icelake, input size: 199331, iterations: 30000, dataset: unicode_lipsum/wikipedia_mars/german.latin1.txt
   0.318 ins/byte,    0.136 cycle/byte,   22.917 GB/s (3.0 %),     3.111 GHz,    2.340 ins/cycle 
   0.318 ins/char,    0.136 cycle/char,   22.917 Gc/s (3.0 %)     1.00 byte/char 
convert_latin1_to_utf8+icelake, input size: 271743, iterations: 30000, dataset: unicode_lipsum/wikipedia_mars/portuguese.latin1.txt
   0.406 ins/byte,    0.160 cycle/byte,   19.369 GB/s (1.8 %),     3.104 GHz,    2.533 ins/cycle 
   0.406 ins/char,    0.160 cycle/char,   19.369 Gc/s (1.8 %)     1.00 byte/char 

This PR:

convert_latin1_to_utf8+icelake, input size: 199331, iterations: 30000, dataset: unicode_lipsum/wikipedia_mars/german.latin1.txt
   0.323 ins/byte,    0.137 cycle/byte,   22.651 GB/s (2.6 %),     3.112 GHz,    2.349 ins/cycle 
   0.323 ins/char,    0.137 cycle/char,   22.651 Gc/s (2.6 %)     1.00 byte/char 
convert_latin1_to_utf8+icelake, input size: 271743, iterations: 30000, dataset: unicode_lipsum/wikipedia_mars/portuguese.latin1.txt
   0.414 ins/byte,    0.165 cycle/byte,   18.838 GB/s (1.4 %),     3.104 GHz,    2.514 ins/cycle 
   0.414 ins/char,    0.165 cycle/char,   18.838 Gc/s (1.4 %)     1.00 byte/char 

See also #347

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant