Installing CI App

This page will walk you through installing CI App. Before you begin, you should have received:

• the CI App on-prem installer (ci-fuzz-on-prem-installer-<version>-linux)
• a docker image bundle (ci-fuzz-on-prem-images-<version>-linux)

Table of contents

• Prerequisites
• Step 1: Get a TLS certificate
• Step 2: Run the installer
• Step 3: Configuration
  • 3.1 First Time Install
  • 3.2 Upgrading CI App
• Step 4: Run server

Prerequisites

In order to use the provided install mechanism and the resulting docker-compose setup, some prerequisites have to be met. We recommend using a server with at least:

• 32 GB RAM
• 16 or more CPU cores
• 100 GB storage
• A Linux operating system
• ssh access (or equivalent)
• Docker and Docker-Compose
• root privileges (if you want to use port 80/443)
• Open ports. See Ports and Network Settings for details
• Access to a docker registry.

Linux

The only supported operating system is Linux. Most linux distributions will be fine, while the most comfort can be achieved by using one that supports systemd. If in doubt, we recommend using Ubuntu Server.

TLS

Operating system’s certificate store must be in the following location:

/etc/ssl/certs/ca-certificates.crt

This is the default on Ubuntu and Debian. On Centos, you need to create a hard link here:

ln /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt /etc/ssl/certs/ca-certificates.crt

Docker

Docker-Compose relies on docker being installed on the host system. To do so, follow the provided instructions for your distribution as depicted in the docker engine documentation.

Additionally, CI App runs fuzz tests inside specific Docker containers. These docker containers are specified as part of the fuzz tests themselves, but the CI App server must have access to these docker images. They can either be pulled from a docker registry (internal or external) or otherwise placed on the CI App server.

Docker-Compose

In addition to the docker engine, the setup uses docker-compose to orchestrate its components. To setup docker-compose, follow the installation instructions.

The minimum supported docker-compose version is 1.27.0. CI App also supports the Compose plugin for Docker (docker compose).

Systemd

Having docker-compose allows to easily start and stop the service. But to be (re)started at boot, additional work has to be done. The installer provides a shortcut by setting up a systemd system service.

Ports and Network Settings

This table provides some details about potential port requirements that may be needed.

AUT is the Application Under Test and applies to Web API fuzzing only.

A DNS entry for your CI App server must exist, or you need a custom TLS certificate that is valid for an IP address.

Step 1: Get a TLS certificate

For secure TLS connections to the fuzzing server, a TLS certificate is needed. Your company may already use a specific CA or run its own. If not, you can get free TLS certificates from Let’s Encrypt.

Just install the command line tool certbot and request a certificate with a single command:

sudo apt install certbot  
sudo certbot certonly --standalone --preferred-challenges http -d <your domain>

Step 2: Run the installer

You should have received a link for the installer and a docker image bundle (CI App on-premise backend image bundle) from Code Intelligence. Run the installer with the docker image bundle as shown here:

chmod +x ci-fuzz-on-prem-installer-<version>-linux
./ci-fuzz-on-prem-installer-<version>-linux --docker-images <path to your bundle>

You need at least 10 GB of free disk space to run the installer when using the image bundle, as it needs to create some temporary files. 

Step 3: Configuration

When CI App is started it will load all configuration variables found in /opt/ci-fuzz-<version>/cifuzz.env. It will then search for any .env files in /etc/cifuzz. Any variables defined there will overwrite those originally taken from /opt/ci-fuzz-<version>/cifuzz.env. Below we provide a template with some explanations and recommended values you can use as a starting point for your configuration.

Data Storage Locations

The following table lists the standard locations and components for persisted data for a CI App installation. This is a useful reference when planning your initial installation and how you will handle upgrades and backups.

If deploying CI App to a new system / image, it is recommended you first start CI App to initialize the docker volume directory. Then stop CI App, copy over the database backup, and restart CI App.

Pulling Docker Images

CI App runs fuzz tests in docker containers. By default, it will attempt to pull docker images from Docker Hub for images that do not already exist on the server. It can also be configured to pull from private docker registries.

To do this, add the following variable to your CI App configuration file : CIFUZZ_JOB_IMAGE_PULL_SECRETS. The value for this variable should be formatted as an edn map of registry name to auth data.

An example of specifying the variable:

CIFUZZ_JOB_IMAGE_PULL_SECRETS='{"myregistry.mycomp.io" {"username" "some-user" "password" "some-password" "serveraddress" "myregistry.mycomp.io"} "myotherregistry.mycomp.io" {"username" "some-other-user" "password" "some-other-password" "serveraddress" "myotherregistry.mycomd.io"}}'

The value should be properly escaped for bash .env files and given in a single line. Double quotes are required and the serveraddress given must match the registry name. If no credentials are configured for a given registry, pulls will be performed without authentication. The system will not try to acces registries for images already available locally.

3.1 First Time Install

If this is your first time installing CI App, you will need to create the /etc/cifuzz directory.

sudo mkdir /etc/cifuzz/

You can download this template to /etc/cifuzz/cifuzz.env and then fill in the values as described below. You can use the command:

sudo wget https://docs.code-intelligence.com/assets/files/cifuzz.env.template -O /etc/cifuzz/cifuzz.env

• CIFUZZ_ARTIFACT_STORAGE_QUOTA - the maximum allowed storage space the artifacts are allowed to consume, before old artifacts get deleted. The default is 100 GB, but depending on the total storage available to the server hosting CI App, you may want to adjust this.
• CIFUZZ_SERVER_ORIGIN - change to the hostname of the host where you are running CI App server. Please set full origin here - protocol (https), hostname and port. If you are using OAuth, then this must match the origin set in your OAuth application.
• CIFUZZ_SERVER_PORT you can change the port if needed. Defaults to 8080 if not set.
• CIFUZZ_CERT_FILE and CIFUZZ_CERT_KEY - absolute paths to the certificate and key that CI App should use.
• CIFUZZ_INTER_SERVICE_AUTH_KEY - generate a random key using openssl rand -hex 32 or equivalent.
• CIFUZZ_NUM_WORKER_THREADS - set to the number of CPU cores available.
• CI App Authentication - there are several ways to configure authentication to the CI App Web Application. The template contains areas for various OAuth applications. You only need to fill in the values for one of them. See Authentication for details on different authentication options.

3.2 Upgrading CI App

If you are upgrading CI App from a previous version, there is no need to adjust your configuration options in /etc/cifuzz/cifuzz.env unless new functionality has been added that you would like to use. To upgrade CI App stop the service, install the new CI App version with the new docker images, and then restart the service:

sudo systemctl stop cifuzz
chmod +x ci-fuzz-on-prem-installer-<version>-linux
./ci-fuzz-on-prem-installer-<version>-linux --docker-images <path to your bundle>
sudo systemctl start cifuzz

You may also need to reload the service files using sudo systemctl daemon-reload after installing the new version of CI App.

Step 4: Run server

sudo systemctl start cifuzz  
sudo systemctl enable cifuzz #to make it autostart after reboot

Check if CI App containers have been started successfully:

sudo docker ps

The output should look similar to this:

CONTAINER ID   IMAGE                                                  COMMAND                  CREATED          STATUS         PORTS                                       NAMES
2833328092d9   f5d6e55758e0d08d.azurecr.io/cifuzz/ci-worker:2.42.0    "/usr/bin/java -cp /…"   11 seconds ago   Up 7 seconds   0.0.0.0:6778->6777/tcp, :::6778->6777/tcp   cifuzz-ci-worker-1
dba452af7e81   f5d6e55758e0d08d.azurecr.io/cifuzz/ci-backend:2.42.0   "/usr/bin/java -cp /…"   11 seconds ago   Up 7 seconds   0.0.0.0:6777->6777/tcp, :::6777->6777/tcp   cifuzz-ci-backend-1
bc3e5af64601   prometheuscommunity/postgres-exporter:v0.10.0          "/bin/postgres_expor…"   11 seconds ago   Up 7 seconds   0.0.0.0:9187->9187/tcp, :::9187->9187/tcp   cifuzz-db-metrics-1
909f0f946f26   f5d6e55758e0d08d.azurecr.io/cifuzz/reporting:2.42.0    "/usr/bin/java -cp /…"   11 seconds ago   Up 8 seconds                                               cifuzz-reporting-1
86d83d710634   postgres:12.7                                          "docker-entrypoint.s…"   11 seconds ago   Up 8 seconds   5432/tcp                                    cifuzz-db-1
6975de4d7388   f5d6e55758e0d08d.azurecr.io/cifuzz/gateway:2.42.0      "/app/cmd/gateway/ba…"   11 seconds ago   Up 8 seconds   0.0.0.0:443->443/tcp, :::443->443/tcp       cifuzz-gateway-1