Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update libfuzzer code to compiler-rt 18 #117

Merged
merged 1 commit into from Feb 19, 2024

Conversation

rchildre3
Copy link
Contributor

Update the LLVM compiler-rt C++ code to the top of the 18.x branch

The summary of the changes from compiler-rt 15 (the last time this code was updated) are:

  1. Improvements to Ctrl+C detection and closing out of a fuzzing session
  2. Loop optimizations, in preventing copy operations
  3. Dynamic pagesize detection
  4. Setting Worker thread names (on Linux)
  5. Use of a bitset data structure to track rarity of features "improves fuzzer throughput rather dramatically (close to 2x) in early exploratory phases"

The LLVM commits which make up the changes in this commit come from the squash of the following LLVM commits:

git log --format=reference origin/release/15.x..release/18.x -- compiler-rt/lib/fuzzer/
3747cde5e84f ([Fuzzer] Enable custom libc++ for Android (#70407), 2023-11-02)
747e0d9f0aad ([compiler-rt] Use std::clamp (NFC), 2023-10-22)
bede46f57eec ([Fuzzer] Optimize UpdateFeatureFrequency (#65288), 2023-09-07)
d3440304b110 ([Fuzzer] Update build.sh to fix build errors (#65496), 2023-09-07)
dd3aa26fc8e9 (Revert "[Fuzzer] SetThreadName implementation for Windows", 2023-08-28)
151e33c768a6 ([NFC][fuzzer] Cleanup SetThreadName after D156317, 2023-08-28)
45eb6026d979 ([fuzzer] Try to fix windows build after D156317, 2023-08-28)
e0e8695a608f (Revert "[fuzzer] Try to fix windows build after D156317", 2023-08-28)
d364597f8a47 ([fuzzer] Try to fix windows build after D156317, 2023-08-28)
cf76ddcbeb10 ([Fuzzer] SetThreadName implementation for Windows, 2023-08-28)
52526065bc80 ([NFC][compiler-rt] Add missing space in libfuzzer -help docs, 2023-07-28)
cb924ddca514 (Revert "[Fuzzer] SetThreadName implementation for Windows", 2023-07-28)
885275bff215 ([Fuzzer] SetThreadName implementation for Windows, 2023-07-27)
f4aa7b5b8d90 (Revert "[Fuzzer] SetThreadName implementation for Windows", 2023-07-27)
e3f935c7f80c ([Fuzzer] SetThreadName implementation for Windows, 2023-07-27)
b2a253855f4e ([Fuzzer] Assign names to workers, 2023-07-26)
9c2f792dceb6 ([fuzzer] Enable loongarch64, 2023-07-18)
42564f97e8ca ([fuzzer][fuchsia] Support RISC-V, 2023-06-15)
facf22b8b07b ([fuzzer] Platfom specific version of PageSize, 2023-05-25)
baa1488c1693 ([fuzzer] Don't hard-code page size in FuzzerUtil.h, 2023-05-25)
cb9f2de2e802 (Revert "[fuzzer] Don't hard-code page size in FuzzerUtil.h", 2023-05-25)
a2b677e81537 ([fuzzer] Don't hard-code page size in FuzzerUtil.h, 2023-05-25)
f98ee40f4b5d ([NFC][Py Reformat] Reformat python files in the rest of the dirs, 2023-05-17)
cdfcf1aa38a4 ([libfuzzer] avoid unneccessary auto-copy, 2023-04-26)
8a986abe98ff ([fuzzer] Fix -Wunreachable-code-aggressive warning in FuzzerDriver.cpp, 2023-04-11)
f2c9d24e8c8d ([compiler-rt] [test] [fuzzer] Don't pass msvc/clang-cl specific flags to mingw tools, 2023-03-31)
90b4d1bcb201 ([fuzzer] Use puts() rather than printf() in CopyFileToErr(), 2023-03-15)
c2df1d8a6d1c ([libfuzzer] add test of cov file-id in control file, 2023-03-22)
d02ff3d57831 (Revert "[fuzzer] Use puts() rather than printf() in CopyFileToErr()", 2023-03-17)
03aa02adb03c ([fuzzer] Use puts() rather than printf() in CopyFileToErr(), 2023-03-15)
0b327814d86c ([libfuzzer] Fix build error due to out-of-line definition of 'Fuzzer' does not match any declaration in 'fuzzer::Fuzzer' (NFC), 2023-03-10)
513d9b9f3d67 ([libfuzzer] avoid unneccessary copy, 2023-03-09)
e09b3be411b0 ([fuzzer] Make sure we never delete Fuzzer, 2023-03-07)
67f5b05cdcdc (Add test for Flags.data_flow_trace, 2023-03-07)
91985c2ee3b3 (Use the right printf format specifiers, 2023-03-07)
6c485409de52 (Adding missing colon, 2023-01-19)
beb3fa2d2efb (Revert "Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests"", 2022-12-13)
255c3e3dcb06 (Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests", 2022-12-13)
68f4ceaf9b4e (Revert "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests", 2022-10-05)
93b1256e38f6 ([compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests, 2022-10-05)
98c2754abd8f ([Libfuzzer] Include signal.h for SIGINT, 2022-09-04)
aa0e9046c16b (Libfuzzer fix for Ctrl + c not working with -fork and -ignore_crashes=1, 2022-08-15)

Update the LLVM compiler-rt C++ code to the top of the 18.x branch

The summary of the changes from compiler-rt 15 (the last time this code
was updated) are:

1. Improvements to Ctrl+C detection and closing out of a fuzzing session
2. Loop optimizations, in preventing copy operations
3. Dynamic pagesize detection
4. Setting Worker thread names (on Linux)
5. Use of a bitset data structure to track rarity of features "improves
   fuzzer throughput rather dramatically (close to 2x) in early
   exploratory phases"

The LLVM commits which make up the changes in this commit come from the
squash of the following LLVM commits:

```sh
git log --format=reference origin/release/15.x..release/18.x -- compiler-rt/lib/fuzzer/
```

```
3747cde5e84f ([Fuzzer] Enable custom libc++ for Android (#70407), 2023-11-02)
747e0d9f0aad ([compiler-rt] Use std::clamp (NFC), 2023-10-22)
bede46f57eec ([Fuzzer] Optimize UpdateFeatureFrequency (#65288), 2023-09-07)
d3440304b110 ([Fuzzer] Update build.sh to fix build errors (#65496), 2023-09-07)
dd3aa26fc8e9 (Revert "[Fuzzer] SetThreadName implementation for Windows", 2023-08-28)
151e33c768a6 ([NFC][fuzzer] Cleanup SetThreadName after D156317, 2023-08-28)
45eb6026d979 ([fuzzer] Try to fix windows build after D156317, 2023-08-28)
e0e8695a608f (Revert "[fuzzer] Try to fix windows build after D156317", 2023-08-28)
d364597f8a47 ([fuzzer] Try to fix windows build after D156317, 2023-08-28)
cf76ddcbeb10 ([Fuzzer] SetThreadName implementation for Windows, 2023-08-28)
52526065bc80 ([NFC][compiler-rt] Add missing space in libfuzzer -help docs, 2023-07-28)
cb924ddca514 (Revert "[Fuzzer] SetThreadName implementation for Windows", 2023-07-28)
885275bff215 ([Fuzzer] SetThreadName implementation for Windows, 2023-07-27)
f4aa7b5b8d90 (Revert "[Fuzzer] SetThreadName implementation for Windows", 2023-07-27)
e3f935c7f80c ([Fuzzer] SetThreadName implementation for Windows, 2023-07-27)
b2a253855f4e ([Fuzzer] Assign names to workers, 2023-07-26)
9c2f792dceb6 ([fuzzer] Enable loongarch64, 2023-07-18)
42564f97e8ca ([fuzzer][fuchsia] Support RISC-V, 2023-06-15)
facf22b8b07b ([fuzzer] Platfom specific version of PageSize, 2023-05-25)
baa1488c1693 ([fuzzer] Don't hard-code page size in FuzzerUtil.h, 2023-05-25)
cb9f2de2e802 (Revert "[fuzzer] Don't hard-code page size in FuzzerUtil.h", 2023-05-25)
a2b677e81537 ([fuzzer] Don't hard-code page size in FuzzerUtil.h, 2023-05-25)
f98ee40f4b5d ([NFC][Py Reformat] Reformat python files in the rest of the dirs, 2023-05-17)
cdfcf1aa38a4 ([libfuzzer] avoid unneccessary auto-copy, 2023-04-26)
8a986abe98ff ([fuzzer] Fix -Wunreachable-code-aggressive warning in FuzzerDriver.cpp, 2023-04-11)
f2c9d24e8c8d ([compiler-rt] [test] [fuzzer] Don't pass msvc/clang-cl specific flags to mingw tools, 2023-03-31)
90b4d1bcb201 ([fuzzer] Use puts() rather than printf() in CopyFileToErr(), 2023-03-15)
c2df1d8a6d1c ([libfuzzer] add test of cov file-id in control file, 2023-03-22)
d02ff3d57831 (Revert "[fuzzer] Use puts() rather than printf() in CopyFileToErr()", 2023-03-17)
03aa02adb03c ([fuzzer] Use puts() rather than printf() in CopyFileToErr(), 2023-03-15)
0b327814d86c ([libfuzzer] Fix build error due to out-of-line definition of 'Fuzzer' does not match any declaration in 'fuzzer::Fuzzer' (NFC), 2023-03-10)
513d9b9f3d67 ([libfuzzer] avoid unneccessary copy, 2023-03-09)
e09b3be411b0 ([fuzzer] Make sure we never delete Fuzzer, 2023-03-07)
67f5b05cdcdc (Add test for Flags.data_flow_trace, 2023-03-07)
91985c2ee3b3 (Use the right printf format specifiers, 2023-03-07)
6c485409de52 (Adding missing colon, 2023-01-19)
beb3fa2d2efb (Revert "Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests"", 2022-12-13)
255c3e3dcb06 (Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests", 2022-12-13)
68f4ceaf9b4e (Revert "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests", 2022-10-05)
93b1256e38f6 ([compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests, 2022-10-05)
98c2754abd8f ([Libfuzzer] Include signal.h for SIGINT, 2022-09-04)
aa0e9046c16b (Libfuzzer fix for Ctrl + c not working with -fork and -ignore_crashes=1, 2022-08-15)
```
@Manishearth Manishearth merged commit 9848925 into rust-fuzz:main Feb 19, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants