This section covers how to run a fuzz test using CI Fuzz CLI. CI Fuzz CLI provides two different approaches to running fuzz tests: running as an actual fuzz test and running as a regression tests. Details for each of these approaches are discussed here.
Fuzz tests you have created can be run using
cifuzz run <fuzz_test>. When you run a fuzz test this way:
- The fuzzing engine will rapidly generate new inputs to pass to the software under test.
- When one of these inputs reaches a new part of the code, it is added to the generated corpus.
- The fuzzing run will continue generating new inputs and attempting to improve code coverage until either a crash occurs or the
--timeoutis reached if you set this value.
- When a crash occurs, you will be notified and a finding will be created in a directory named
<fuzz_test>_inputs. This directory is created adjacent to the location of the fuzz test.
When running as a regression fuzz test,
cifuzz will only use existing inputs that are stored in the
<fuzz_test>_inputs directory. These are the inputs that previously led to findings. A regression fuzz test will run until all of these inputs are test or a regression (crash) occurs. In this mode,
cifuzz will not generate any new inputs, so regression fuzz testing will typically be quite fast and enable you to rapidly check for any regressions.
To create a regression fuzz test run, you can:
- Create a replayer binary that can be invoked through the command line. The sections below describe how to create these replayer binaries for different build systems.
- Integrate CI Fuzz CLI with your IDE.
For CMake projects, you can use the CMake user presets that CI Fuzz CLI provides. These can be generated by running the following command in the root of the project:
cifuzz integrate cmake
This creates a
CMakeUserPresets.json in the root directory of the project. With the presets created, you can run regression fuzz tests directly through
CMake by using the following:
cmake --preset="cifuzz (Regression Test)" cmake --build --preset="cifuzz (Regression Test)" ctest --preset="cifuzz (Regression Test)"
For Bazel projects, you must add the following configuration to the
.bazelrc file in the project workspace:
build:cifuzz-replay --@rules_fuzzing//fuzzing:cc_engine=@rules_fuzzing//fuzzing/engines:replay build:cifuzz-replay --@rules_fuzzing//fuzzing:cc_engine_instrumentation=none build:cifuzz-replay --@rules_fuzzing//fuzzing:cc_engine_sanitizer=asan build:cifuzz-replay --@rules_fuzzing//fuzzing:cc_engine_sanitizer=ubsan build:cifuzz-replay --compilation_mode=opt build:cifuzz-replay --copt=-g build:cifuzz-replay --copt=-U_FORTIFY_SOURCE build:cifuzz-replay --test_env=UBSAN_OPTIONS=halt_on_error=1
This allows the bazel flag
--config=cifuzz-replay to be added to a fuzz test run via
bazel test --config=cifuzz-replay.