Run fuzzing on OEMCrypto public APIs on linux using google supported clusterfuzz infrastructure to find security vulnerabilities.

Design Document - https://docs.google.com/document/d/1mdSV2irJZz5Y9uYb5DmSIddBjrAIZU9q8G5Q_BGpA4I/edit?usp=sharing

Fuzzing at google - go/fuzzing

Monitoring

Cluster fuzz statistics

Performance of OEMCrypto fuzz binaries running continuously using cluster fuzz infrastructure can be monitored here.

The options to select are Job type: libfuzzer_asan_oemcrypto and Fuzzer: fuzzer name you are looking for

Example: load_license_fuzz

Issues filed by clusterfuzz - Fixing those issues

Any issues found with the fuzz target under test are reported by clusterfuzz here.
The bug will have a link to the test case that generated the bug. Download the test case and follow the steps from testing fuzzer locally section to run the fuzzer locally using the test case that caused the crash.
Once the issue is fixed, consider adding the test case that caused the crash to the seed corpus zip file. Details about seed corpus and their location are mentioned in this section.

Corpus

Once the fuzzer scripts are ready and running continuously using clusterfuzz or android infrastructure, we can measure the efficiency of fuzzers by looking at code coverage and number of new features that have been discovered by fuzzer scripts here Fuzz script statistics.

A fuzzer which tries to start from random inputs and figure out intelligent inputs to crash the libraries can be time consuming and not effective. A way to make fuzzers more effective is by providing a set of valid and invalid inputs of the library so that fuzzer can use those as a starting point. These sets of valid and invalid inputs are called corpus.

The idea is to run OEMCrypto unit tests and read required data into binary corpus files before calling into respective OEMCrypto APIs under test. Writing corpus data to binary files is controlled by --generate_corpus flag.

Build OEMCrypto unit tests to generate corpus

Install Pre-requisites
```
$ sudo apt-get install gyp ninja-build
```
download cdm source code (including ODK & OEMCrypto unit tests):
```
$ git clone sso://widevine-internal/cdm
```

Build OEMCrypto unit tests and run with --generate_corpus flag to generate corpus files:

$ cd /path/to/cdm/repo
$ export CDM_DIR=/path/to/cdm/repo
$ export PATH_TO_CDM_DIR=..
$ gyp --format=ninja --depth=$(pwd) oemcrypto/oemcrypto_unittests.gyp
$ ninja -C out/Default/
$ ./out/Default/oemcrypto_unittests --generate_corpus

To avoid uploading huge binary files to git repository, the corpus files will be saved in fuzzername_seed_corpus.zip format in blockbuster project's oemcrypto_fuzzing_corpus GCS bucket using gsutil. If you need permissions for blockbuster project, contact widevine-engprod@google.com.
```
$ gsutil cp gs://oemcrypto_fuzzing_corpus/<fuzzername_seed_corpus.zip> \
    <destination_path>
```

Testing fuzzer locally

Corpus needed to run fuzz tests locally are available in blockbuster project's oemcrypto_fuzzing_corpus GCS bucket. If you need permissions for this project, contact widevine-engprod@google.com. Download corpus.
```
$ gsutil cp gs://oemcrypto_fuzzing_corpus/<fuzzername_seed_corpus.zip> \
  <destination_path>
```
Add flags to generate additional debugging information. Add '-g3' flag to oemcrypto_fuzztests.gypi cflags_cc in order to generate additional debug information locally.

Build and test fuzz scripts locally using:

$ export CXX=clang++
$ export CC=clang
$ export GYP_DEFINES="clang=1"
$ cd /path/to/cdm/repo
$ export PATH_TO_CDM_DIR=.
$ gyp --format=ninja --depth=$(pwd) \
    oemcrypto/test/fuzz_tests/oemcrypto_fuzztests.gyp
$ ninja -C out/Default/
$ mkdir /tmp/new_interesting_corpus
$ ./out/Default/fuzzer_binary /tmp/new_interesting_corpus \
    /path/to/fuzz/seed/corpus/folder

In order to run fuzz script against a crash input, follow the above steps and run the fuzz binary against crash input rather than seed corpus.
```
$ ./out/Default/fuzzer_binary crash_input_file
```

Adding a new OEMCrypto fuzz script

In order to fuzz a new OEMCrypto API in future, a fuzz script can be added to oemcrypto/test/fuzz_tests folder which ends with _fuzz.cc.
In the program, define the function LLVMFuzzerTestOneInput with the following signature:
```
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
    <your test code goes here>
    return 0;
}
```
Note: Make sure LLVMFuzzerTestOneInput calls the function you want to fuzz.
Add a new target to oemcrypto_fuzztests.gyp file and follow instructions in testing fuzzer locally to build and test locally.

Generate code coverage reports locally

Code coverage is a means of measuring fuzzer performance. We want to make sure that our fuzzer covers all the paths in our code and make any tweeks to fuzzer logic so we can maximize coverage to get better results.

Coverage reports for git on borg project is not automated and needs to be generated manually. Future plan is to build a dashboard for git on borg coverage reports.
In order to generate coverage reports, we need to compile fuzzer binary with flags to enable coverage. We can remove -fsanitize=fuzzer,address,undefined from oemcrypto_fuzztests.gypi file as that is needed only while fuzzing. Add following flags to both cflags_cc and ldflags of oemcrypto_fuzztests.gypi and build fuzz binaries as mentioned in Testing fuzzer locally section.
```
'-fprofile-instr-generate',
'-fcoverage-mapping',
```
We need to run fuzzer binary against the corpus downloaded from clusterfuzz. Clock on download link from corpus_backup column. Use gsutil command to download the entire corpus for the fuzz binary.

Use the following commands to generate raw profile data file with coverage information and generate a html coverage report for a single fuzzer. More information about clang source based coverage can be found here. Follow this for steps to combine code coverage reports of multiple fuzzers.

# Run fuzz binary against corpus backup to generate default.profraw file.
$ ./out/Default/fuzz_binary path/to/corpus/backup -runs=0
# Index raw profile files to generate coverage reports.
$ llvm-profdata merge -sparse default.profraw -o default.profdata
# Generate html coverage file.
$ llvm-cov show ./out/Default/fuzz_binary -format=html \
-instr-profile=default.profdata -o default.html