Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix buffer overflows (CVE-2021-45958) #519

Merged
merged 11 commits into from Apr 5, 2022
Merged

Fix buffer overflows (CVE-2021-45958) #519

merged 11 commits into from Apr 5, 2022

Commits on Apr 5, 2022

  1. Fix unchecked buffer overflows (CVE-2021-45958). 

    Add a few extra memory reserve calls to account for the extra space that
indentation needs.

These kinds of memory issues are hard to spot because the buffer is resized in
powers of 2 meaning that a miscalculation would only show any symptoms if the
required buffer size is estimated to be just below a 2 power but is actually
just above. Add a debug mode which replaces the 2 power scheme with reserving
only the memory explicitly requested and adds some overflow checks.
    @bwoodsend @JustAnotherArchivist
    bwoodsend authored and JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    60929c2 View commit details
    Browse the repository at this point in the history

  2. Add a fuzzing test to search for segfaults in encoding.

    @bwoodsend @JustAnotherArchivist
    bwoodsend authored and JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    62e4853 View commit details
    Browse the repository at this point in the history

  3. Remove the hidden JSON_NO_EXTRA_WHITESPACE compile knob. 

    Unsetting it can lead to seg-faults. I don't think it's worth having to fix and
then test this undocumented permutation.
    @bwoodsend @JustAnotherArchivist
    bwoodsend authored and JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    22ca5dc View commit details
    Browse the repository at this point in the history

  4. Fix some more seg-faults on encoding.

    @bwoodsend @JustAnotherArchivist
    bwoodsend authored and JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    3151be2 View commit details
    Browse the repository at this point in the history

  5. Add fuzz test to CI/CD.

    @bwoodsend @JustAnotherArchivist
    bwoodsend authored and JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    fdfb9f4 View commit details
    Browse the repository at this point in the history

  6. Refactor buffer reservations to ensure sufficient space on all additions 

    * Removed the reservations in Buffer_EscapeStringUnvalidated and Buffer_EscapeStringValidated as those are not needed and may hide other bugs.
* Debug check in Buffer_EscapeStringValidated was triggering incorrectly.
* The reservation on JT_RAW was much larger than necessary; the value is copied directly, so the factor six is not needed, and this may hide other bugs.
* Explicit accurate reservations everywhere else.
    @JustAnotherArchivist
    JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    7039d60 View commit details
    Browse the repository at this point in the history

  7. Widen tests to cover more possible buffer overflows 

    If the default output format changes in the future (e.g. `separators` as in the standard library), these tests would otherwise become irrelevant.
    @JustAnotherArchivist
    JustAnotherArchivist committed Apr 5, 2022
    Copy the full SHA
    36ffcc8 View commit details
    Browse the repository at this point in the history

  8. actions/checkout@v3 

    Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
    @JustAnotherArchivist @hugovk
    JustAnotherArchivist and hugovk committed Apr 5, 2022
    Copy the full SHA
    968fb16 View commit details
    Browse the repository at this point in the history

  9. Clearer pytest command 

    Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
    @JustAnotherArchivist @hugovk
    JustAnotherArchivist and hugovk committed Apr 5, 2022
    Copy the full SHA
    3944dae View commit details
    Browse the repository at this point in the history

  10. python -m pytest 

    Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
    @JustAnotherArchivist @hugovk
    JustAnotherArchivist and hugovk committed Apr 5, 2022
    Copy the full SHA
    9b50121 View commit details
    Browse the repository at this point in the history

  11. Remove shebang 

    Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
    @JustAnotherArchivist @hugovk
    JustAnotherArchivist and hugovk committed Apr 5, 2022
    Copy the full SHA
    7dd7429 View commit details
    Browse the repository at this point in the history