Skip to main content

Find Your First Bug in Java

This quick start guide will teach you the basics of using CI Fuzz with a Java project. CI Fuzz supports Maven and Gradle as build systems.

This guide was created using macOS. If you are on Linux or Windows, you should be able to follow along without any issues.

If you did not already install cifuzz, head over to the setup section.

Maven​

1. Set Up the Repository​

Download or clone the following repository: https://github.com/CodeIntelligenceTesting/ci-fuzz-cli-tutorials. This repository contains several tutorials and example projects. For this guide, we will use the project in the tutorials/java/maven directory.

note

Run all cifuzz commands listed in this section from the tutorials/java/maven directory.

2. Initialize the Project​

To use CI Fuzz in the project, you first need to initialize it.

Run the following command to create the CI Fuzz configuration file cifuzz.yaml:

cifuzz init

CI Fuzz will the project as a Maven project and provide instructions how to configure your project to enable fuzz testing.

For an easy setup we developed our own CI Fuzz Maven extension, which you have to add to your projects pom.xml additionally with the access to our CI repository:

pom.xml
<build>
    <extensions>
        <extension>
            <groupId>com.code-intelligence</groupId>
            <artifactId>cifuzz-maven-extension</artifactId>
            <version>1.7.0</version>
        </extension>
    </extensions>
</build>

<repositories>
    <repository>
        <id>code-intelligence</id>
        <url>https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven</url>
    </repository>
</repositories>
<pluginRepositories>
    <pluginRepository>
        <id>code-intelligence</id>
        <url>https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven</url>
    </pluginRepository>
</pluginRepositories>

As stated in the instructions, don't forget to add your access information to the CI Repository in your global settings file ~/.m2/settings.xml:

settings.xml
<?xml version="1.0" encoding="UTF-8"?>
    <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 http://maven.apache.org/xsd/settings-1.2.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.2.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <servers>
            <server>
                <id>code-intelligence</id>
                <username><!--YOUR_USERNAME--></username>
                <password><!--YOUR_TOKEN--></password>
            </server>
    </servers>
</settings>

Following the setup instructions, your pom.xml should look like this:

pom.xml
<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.github.CodeIntelligenceTesting.cifuzz</groupId>
  <artifactId>maven-example</artifactId>
  <version>1.0-SNAPSHOT</version>

  <name>maven-example</name>
  <description>A simple maven-example for cifuzz</description>

  <properties>
    <maven.compiler.source>8</maven.compiler.source>
    <maven.compiler.target>8</maven.compiler.target>
  </properties>

  <dependencies>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.1</version>
      </plugin>
    </plugins>
    <extensions>
      <extension>
        <groupId>com.code-intelligence</groupId>
        <artifactId>cifuzz-maven-extension</artifactId>
        <version>1.7.0</version>
      </extension>
    </extensions>
  </build>

  <repositories>
    <repository>
      <id>code-intelligence</id>
      <url>https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven</url>
    </repository>
  </repositories>
  <pluginRepositories>
    <pluginRepository>
      <id>code-intelligence</id>
      <url>https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven</url>
    </pluginRepository>
  </pluginRepositories>
</project>

3. Create a Fuzz Test​

The next step is to create a Java Fuzz Test template. To make fuzz testing as easy as unit testing, you should place the Fuzz Test in a test directory, exactly as you would a standard unit test.

Run the following command to create a file MyFuzzTest.java with a Fuzz Test stub:

cifuzz create java -o src/test/java/com/example/MyFuzzTest.java

Before you write the Fuzz Test, take a look at the target method that you want to fuzz in src/main/java/com/example/ExploreMe.java:

ExploreMe.java
public static void exploreMe(int a, int b, String c) {
    if (a >= 20000) {
        if (b >= 2000000) {
            if (b - a < 100000) {
                if (c.startsWith("@")) {
                    String className = c.substring(1);
                    try {
                        Class.forName(className).newInstance();
                    } catch (ClassNotFoundException | InstantiationException | IllegalAccessException ignored) {
                    }
                }
            }
        }
    }
}

The main parts to focus on here is the parameters that the exploreMe function requires: int a, int b and string c. As long as you pass the correct data types to the function, the fuzzer takes care of the rest.

Now you can fill in the Fuzz Test template with a real Fuzz Test:

MyFuzzTest.java
package com.example;

import com.code_intelligence.jazzer.api.FuzzedDataProvider;
import com.code_intelligence.jazzer.junit.FuzzTest;

public class MyFuzzTest {
    @FuzzTest
    void myFuzzTest(FuzzedDataProvider data) {
        int a = data.consumeInt();
        int b = data.consumeInt();
        String c = data.consumeRemainingAsString();

        ExploreMe.exploreMe(a, b, c);
    }
}

A few notes about this Fuzz Test:

  • The Fuzz Test is part of the same package as the target class/method.
  • The @FuzzTest annotation is what enables you to write Fuzz Tests similar to how you'd write JUnit tests.
  • The Fuzz Test must import com.code_intelligence.jazzer.junit.FuzzTest
  • This Fuzz Test uses the FuzzedDataProvider class. This isn't required, but it's a convenient way to split the fuzzing input in the data variable into different data types. Here is a link to the FuzzedDataProvider class documentation if you want to view its other methods.
  • Once you have created the appropriate variables (a, b, and c) using data from the fuzzer, the Fuzz Test has to call the target method (ExploreMe.exploreMe) with the fuzz data.

4. Run the Fuzz Test​

You now have successfully set up and initialized your project, added all necessary configuration and created a Fuzz Test.

Now you can run the Fuzz Test:

cifuzz run com.example.MyFuzzTest::myFuzzTest

cifuzz should discover a Remote Code Execution quickly:

5 Unit Test Equivalents and 1 new Finding in 6s.
1 Finding in total.

💥 [nimble_peccary] Security Issue: Remote Code Execution in com.example.ExploreMe.exploreMe (src/main/java/com/example/ExploreMe.java:11)

The command will also ask if you want to run the cifuzz coverage command, in case you are interested in coverage information of the Fuzz Test.

5. Examine the Finding​

When cifuzz discovers a Finding, it stores the output from the Finding and the input that caused it. You can list all Findings discovered so far by running cifuzz findings.

Origin | Severity | Name           | Description                           | Fuzz Test              | Location
Local  | 7.0      | nimble_peccary | Security Issue: Remote Code Execution | com.example.MyFuzzTest | src/main/java/com/example/ExploreMe.java:11

If you want to see the stack trace and details of a specific Finding you can provide its name, for example cifuzz finding nimble_peccary. This information can help you debug and fix the issue.

Examining the output from cifuzz finding nimble_peccary shows that there was a Remote Code Execution and this triggers at line 11 in the exploreMe function. If you examine line 11 in src/main/java/com/example/ExploreMe.java, you can see this is where it attempts to load a class based off of input data.

 == Java Exception: com.code_intelligence.jazzer.api.FuzzerSecurityIssueHigh: Remote Code Execution
  Unrestricted class/object creation based on externally controlled data may allow
  remote code execution depending on available classes on the classpath.
        at jaz.Zer.reportFinding(Zer.java:108)
        at jaz.Zer.reportFindingIfEnabled(Zer.java:103)
        at jaz.Zer.<init>(Zer.java:76)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
        at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:347)
        at java.base/java.lang.Class.newInstance(Class.java:645)
        at com.example.ExploreMe.exploreMe(ExploreMe.java:11)
        at com.example.MyFuzzTest.myFuzzTest(MyFuzzTest.java:13)

If possible, CI Fuzz will also provide more information about the kind of Finding:

┌────────────────────────────────────────────────────────────────────────────────────────────────────┐
| Name            | Remote Code Execution                                                            |
| Severity Level  | Critical                                                                         |
| Severity Score  | 7.0                                                                              |
| CWE Name        | Improper Control of Generation of Code ('Code Injection')                        |
| CWE Description | The product constructs all or part of a code segment using                       |
|                 | externally-influenced input from an upstream component, but it does not          |
|                 | neutralize or incorrectly neutralizes special elements that could modify the     |
|                 | syntax or behavior of the intended code segment when it is sent to a downstream  |
|                 | component.                                                                       |
└────────────────────────────────────────────────────────────────────────────────────────────────────┘
Description:
Remote code execution attacks allow an attacker to execute arbitrary code on a vulnerable system. 
This can be used to steal data, bypass authentication, and gain access to more systems. 
RCE is often combined with other application vulnerability, such as SQL injection, to gain initial access to a system.

cifuzz stores the crashing input in the resources directory. In this case: src/test/resources/com/example/MyFuzzTestInputs/myFuzzTest. The crashing input has the same name as the Finding and Findings in this directory will serve as inputs for future runs to help identify regressions.

Gradle​

1. Set Up the Repository​

Download or clone the following repository: https://github.com/CodeIntelligenceTesting/ci-fuzz-cli-tutorials. This repository contains several tutorials and example projects. For this guide, we will use the project in the tutorials/java/gradle directory.

note

Run all cifuzz commands listed in this section from the tutorials/java/gradle directory.

2. Initialize the Project​

To use CI Fuzz in the project, you first need to initialize it.

Run the following command to create the CI Fuzz configuration file cifuzz.yaml:

cifuzz init

CI Fuzz will the project as a Gradle project and provide instructions how to configure your project to enable fuzz testing.

For an easy setup we developed our own CI Fuzz Gradle plugin, which you have to add to your projects build.gradle additionally with the access to our CI repository:

plugins {
    id "com.code-intelligence.cifuzz" version "1.17.0"
}

repositories {
    maven {
        name "CodeIntelligenceRepository"
        url "https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven"
        credentials {
            username CodeIntelligenceRepositoryUsername
            password CodeIntelligenceRepositoryPassword
        }
        content {
            includeGroupByRegex("com\\.code-intelligence.*")
        }
    }
    mavenCentral()
}

As stated in the instructions, don't forget to add your access information to the CI Repository in your global settings file ~/.gradle/gradle.properties and the projects settings.gradle:

CodeIntelligenceRepositoryUsername=<YOUR_USERNAME>
CodeIntelligenceRepositoryPassword=<YOUR_TOKEN>
pluginManagement {
    repositories {
        maven {
            name "CodeIntelligenceRepository"
            url "https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven"
            credentials {
                username CodeIntelligenceRepositoryUsername
                password CodeIntelligenceRepositoryPassword
            }
            content {
              includeGroupByRegex("com\\.code-intelligence.*")
            }
        }
        gradlePluginPortal()
    }
}

Following the setup instructions, your build.gradle should look like this:

plugins {
    id 'application'
    id "com.code-intelligence.cifuzz" version "1.17.0"
}

repositories {
    maven {
        name "CodeIntelligenceRepository"
        url "https://gitlab.code-intelligence.com/api/v4/projects/89/packages/maven"
        credentials {
            username CodeIntelligenceRepositoryUsername
            password CodeIntelligenceRepositoryPassword
        }
        content {
            includeGroupByRegex("com\\.code-intelligence.*")
        }
    }
    mavenCentral()
}

application {
    mainClass = 'com.example.App'
}

3. Create a Fuzz Test​

The next step is to create a Java Fuzz Test template. To make fuzz testing as easy as unit testing, you should place the Fuzz Test in a test directory, exactly as you would a standard unit test.

Run the following command to create a file MyFuzzTest.java with a Fuzz Test stub:

cifuzz create java -o src/test/java/com/example/MyFuzzTest.java

Before you write the Fuzz Test, take a look at the target method that you want to fuzz in src/main/java/com/example/ExploreMe.java:

ExploreMe.java
public static void exploreMe(int a, int b, String c) {
  if (a >= 20000) {
    if (b >= 2000000) {
      if (b - a < 100000) {
        if (c.startsWith("@")) {
          String className = c.substring(1);
          try {
            Class.forName(className).newInstance();
          } catch (ClassNotFoundException | InstantiationException | IllegalAccessException ignored) {
          }
        }
      }
    }
  }
}

TThe main parts to focus on here is the parameters that the exploreMe function requires: int a, int b and string c. As long as you pass the correct data types to the function, the fuzzer takes care of the rest.

Now you can fill in the Fuzz Test template with a real Fuzz Test:

MyFuzzTest.java
package com.example;

import com.code_intelligence.jazzer.api.FuzzedDataProvider;
import com.code_intelligence.jazzer.junit.FuzzTest;

public class MyFuzzTest {
    @FuzzTest
    void myFuzzTest(FuzzedDataProvider data) {
        int a = data.consumeInt();
        int b = data.consumeInt();
        String c = data.consumeRemainingAsString();

        ExploreMe.exploreMe(a, b, c);
    }
}

A few notes about this Fuzz Test:

  • The Fuzz Test is part of the same package as the target class/method.
  • The @FuzzTest annotation is what enables you to write Fuzz Tests similar to how you'd write JUnit tests.
  • The Fuzz Test must import com.code_intelligence.jazzer.junit.FuzzTest
  • This Fuzz Test uses the FuzzedDataProvider class. This isn't required, but it's a convenient way to split the fuzzing input in the data variable into different data types. Here is a link to the FuzzedDataProvider class documentation if you want to view its other methods.
  • Once you have created the appropriate variables (a, b, and c) using data from the fuzzer, the Fuzz Test has to call the target method (ExploreMe.exploreMe) with the fuzz data.

4. Run the Fuzz Test​

You now have successfully set up and initialized your project, added all necessary configuration and created a Fuzz Test.

Now you can run the Fuzz Test:

cifuzz run com.example.MyFuzzTest::myFuzzTest

cifuzz should discover a Remote Code Execution quickly:

5 Unit Test Equivalents and 1 new Finding in 4s.
1 Finding in total.

💥 [clever_wasp] Security Issue: Remote Code Execution in com.example.ExploreMe.exploreMe (src/main/java/com/example/ExploreMe.java:11)

The command will also ask if you want to run the cifuzz coverage command, in case you are interested in coverage information of the Fuzz Test.

5. Examine the Findings​

When cifuzz discovers a Finding, it stores the output from the Finding and the input that caused it. You can list all Findings discovered so far by running cifuzz findings.

Origin | Severity | Name        | Description                           | Fuzz Test              | Location
Local  | 7.0      | clever_wasp | Security Issue: Remote Code Execution | com.example.MyFuzzTest | src/main/java/com/example/ExploreMe.java:11

If you want to see the stack trace and details of a specific Finding you can provide its name, for example cifuzz finding clever_wasp. This information can help you debug and fix the issue.

Examining the output from cifuzz finding nimble_peccary shows that there was a Remote Code Execution and this triggers at line 11 in the exploreMe function. If you examine line 11 in src/main/java/com/example/ExploreMe.java, you can see this is where it attempts to load a class based off of input data.

 == Java Exception: com.code_intelligence.jazzer.api.FuzzerSecurityIssueHigh: Remote Code Execution
  Unrestricted class/object creation based on externally controlled data may allow
  remote code execution depending on available classes on the classpath.
        at jaz.Zer.reportFinding(Zer.java:108)
        at jaz.Zer.reportFindingIfEnabled(Zer.java:103)
        at jaz.Zer.<init>(Zer.java:76)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
        at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:347)
        at java.base/java.lang.Class.newInstance(Class.java:645)
        at com.example.ExploreMe.exploreMe(ExploreMe.java:11)
        at com.example.MyFuzzTest.myFuzzTest(MyFuzzTest.java:13)

If possible, CI Fuzz will also provide more information about the kind of Finding:

┌────────────────────────────────────────────────────────────────────────────────────────────────────┐
| Name            | Remote Code Execution                                                            |
| Severity Level  | Critical                                                                         |
| Severity Score  | 7.0                                                                              |
| CWE Name        | Improper Control of Generation of Code ('Code Injection')                        |
| CWE Description | The product constructs all or part of a code segment using                       |
|                 | externally-influenced input from an upstream component, but it does not          |
|                 | neutralize or incorrectly neutralizes special elements that could modify the     |
|                 | syntax or behavior of the intended code segment when it is sent to a downstream  |
|                 | component.                                                                       |
└────────────────────────────────────────────────────────────────────────────────────────────────────┘
Description:
Remote code execution attacks allow an attacker to execute arbitrary code on a vulnerable system. 
This can be used to steal data, bypass authentication, and gain access to more systems. 
RCE is often combined with other application vulnerability, such as SQL injection, to gain initial access to a system.

cifuzz stores the crashing input in the resources directory. In this case: src/test/resources/com/example/MyFuzzTestInputs/myFuzzTest. The crashing input has the same name as the Finding and Findings in this directory will serve as inputs for future runs to help identify regressions.