Skip to main content

AI Agent Quickstart

In this quickstart you'll automatically fuzz the open source library libyaml.

note

Before getting started please make sure you can use an LLM. Please see here for a detailed instruction.

$ git clone git@github.com:yaml/libyaml.git

After having cloned the library, you'll initialize the AI Test Agent.

$ cifuzz init

The first step is to enable the AI Test Agent to run fuzz tests.

$ cifuzz init

cifuzz init
cifuzz version 6.4.0
✅ Configuration saved in cifuzz.yaml

==============================================================================================================================
Step 1/4: Setup CI Fuzz to run fuzz tests
==============================================================================================================================

To list, run, and calculate coverage for fuzz tests, CI Fuzz needs to adjust your build setup.
Add to 'CMakeLists.txt':
------------------------------------------------------------------------------------------------------------------------------
find_package(cifuzz NO_SYSTEM_ENVIRONMENT_PATH)
enable_fuzz_testing()
------------------------------------------------------------------------------------------------------------------------------

Do you want CI Fuzz to apply the changes above automatically? [Y/n]Y

Afterwards you'll continue by enabling the AI Test Agent to automatically generate fuzz tests.

Do you want to setup CI Fuzz Spark to automatically generate fuzz tests? [Y/n]Y

===================================================================================================
Step 2/4: Determine necessary link targets
===================================================================================================

CI Fuzz needs a list of all build targets that need to be linked.
 ✅   Successfully enumerated CMake targets!
The following 1 CMake targets were found in your build system: yaml
Deselect any targets that you don't want to link into newly generated fuzz tests:
> [✓] yaml
tab: select | enter: confirm | left: none | right: all| type to filter

CMake automatically found the correct target yaml, which you can select. As there are no additional targets for this example the next question confirm the next question:

Add any additional CMake targets that need to be linked into the newly generated fuzz test. [Separate with space or comma, leave empty for none]:

One thing that's left is to enable the AI Test Agent to automatically add fuzz tests to the build system.

===================================================================================================
Step 3/4: Setup build system to add new fuzz tests automatically
===================================================================================================

To add new fuzz tests, CI Fuzz needs to adjust your build setup.
Create new file 'cifuzz-spark/CMakeLists.txt' with the following contents:
---------------------------------------------------------------------------------------------------
#cifuzz:build-template:begin
#add_fuzz_test({{ .FuzzTestName }} {{ .FileName }})
#target_link_libraries({{ .FuzzTestName }} PRIVATE "$<LINK_LIBRARY:WHOLE_ARCHIVE,yaml>")
#cifuzz:build-template:end
---------------------------------------------------------------------------------------------------

Add to 'CMakeLists.txt':
---------------------------------------------------------------------------------------------------
add_subdirectory(cifuzz-spark)
---------------------------------------------------------------------------------------------------

Do you want CI Fuzz to apply the changes above automatically? [Y/n]Y
✅ Changes have been added to the file 'cifuzz-spark/CMakeLists.txt'
✅ Configuration option 'build-file' has been updated in 'cifuzz.yaml'
✅ Changes have been added to the file 'CMakeLists.txt'

The final step is to select the fuzzing engine.

Which engine would you like to use for fuzzing?:
  > libfuzzer-clang
Persist engine choice for future fuzz tests? [Y/n]Y
Engine 'libfuzzer-clang' saved in cifuzz.yaml.
 ✅   Successfully setup CI Fuzz Spark!
 ✅   CI Fuzz has been successfully set up.

To automatically generate fuzz tests, run cifuzz spark

Now that you have successfully initialized the AI Test Agent for the project, you can start fuzzing.

The next command tells the AI Test Agent to automatically generate fuzz tests until it achieves at least 75% coverage.

$ cifuzz spark --target-coverage 75
 ✅   Validating build system configuration...
 ✅   Configuring CMake project...
 ✅   Analyzing files... 59 candidates found.
 ✅   Fuzz test for "yaml_parser_load"... Total Coverage: 65.85% (3849/5845 lines)
 ✅   Fuzz test for "yaml_emitter_dump"... Total Coverage: 68.74% (4018/5845 lines)
 ✅   Fuzz test for "yaml_emitter_open"... Total Coverage: 70.33% (4111/5845 lines)
 ✅   Fuzz test for "yaml_emitter_emit"... Total Coverage: 75.84% (4433/5845 lines)
Surpassed target coverage of 75%.

     FUNCTION            STATUS       LOCATION
 1   yaml_parser_load    Successful   fuzz-tests/fuzz_yaml_parser_load.cpp
 2   yaml_emitter_dump   Successful   fuzz-tests/fuzz_yaml_emitter_dump.cpp
 3   yaml_emitter_open   Successful   fuzz-tests/fuzz_yaml_emitter_open.cpp
 4   yaml_emitter_emit   Successful   fuzz-tests/fuzz_yaml_emitter_emit.cpp

 🚀 4 successful fuzz tests.
 🚀 0 Findings detected.
 🚀 2.26k Unique Test Cases.
 🚀 75.84% total code coverage.
Increased code coverage (from fuzz tests) from 0.00% to 75.84%

Congratulations! You have successfully fuzzed your first project.

You can now examine the created fuzz tests in the fuzz-tests/ folder or produce a detailed coverage report with cifuzz coverage.