Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace all CVTUTF code #567

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Replace all CVTUTF code #567

wants to merge 3 commits into from

Commits on Feb 23, 2024

  1. Delete code that is based on CVTUTF 

    I did this based on manual inspection, comparing the code to my re-created
history of CVTUTF at https://git.lukeshu.com/2git/cvtutf/ (created by the
scripts at https://git.lukeshu.com/2git/cvtutf-make/)
    @LukeShu
    LukeShu committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    abf962a View commit details
    Browse the repository at this point in the history

  2. Adjust to the CVTUTF code being gone 

    I, Luke T. Shumaker, am the sole author of the added code.

I did not reference CVTUTF when writing it.  I did reference the
Unicode standard (15.0.0), the Wikipedia article on UTF-8, and the
Wikipedia article on UTF-16.  When I saw some tests fail, I did
reference the old deleted code (but a JSON-specific part, inherently
not as based on CVTUTF) to determine that script_safe should also
escape U+2028 and U+2029.

I targeted simplicity and clarity when writing the code--it can likely
be optimized.  In my mind, the obvious next optimization is to have it
combine contiguous non-escaped characters into just one call to
fbuffer_append(), instead of calling fbuffer_append() for each
character.

Regarding the use of the "modern" types `uint32_t`, `uint16_t`, and
`bool`:
 - ruby.h is guaranteed to give us uint32_t and uint16_t.
 - Since Ruby 3.0.0, ruby.h is guaranteed to give us bool... but we
   support down to Ruby 2.3.  But, ruby.h is guaranteed to give us
   HAVE_STDBOOL_H for the C99 stdbool.h; so use that to include
   stdbool.h if we can, and if not then fall back to a copy of the
   same bool definition that Ruby 3.0.5 uses with C89.
    @LukeShu
    LukeShu committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    8720b46 View commit details
    Browse the repository at this point in the history

  3. generator.c: Optimize by combining calls to fbuffer_append

    @LukeShu
    LukeShu committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    6e75be6 View commit details
    Browse the repository at this point in the history