187 lines
7.5 KiB
Markdown
187 lines
7.5 KiB
Markdown
# OEMCRYPTO Fuzzing
|
|
|
|
## Objective
|
|
|
|
* Run fuzzing on OEMCrypto public APIs on linux using google supported
|
|
clusterfuzz infrastructure to find security vulnerabilities.
|
|
|
|
Design Document - https://docs.google.com/document/d/1mdSV2irJZz5Y9uYb5DmSIddBjrAIZU9q8G5Q_BGpA4I/edit?usp=sharing
|
|
|
|
Fuzzing at google -
|
|
[go/fuzzing](https://g3doc.corp.google.com/security/fuzzing/g3doc/fuzzing_resources.md?cl=head)
|
|
## Monitoring
|
|
### Cluster fuzz statistics
|
|
|
|
* Performance of OEMCrypto fuzz binaries running continuously using cluster
|
|
fuzz infrastructure can be monitored
|
|
[here](https://clusterfuzz.corp.google.com/fuzzer-stats).
|
|
|
|
The options to select are `Job type: libfuzzer_asan_oemcrypto` and `Fuzzer:
|
|
fuzzer name you are looking for`
|
|
|
|
Example: [load_license_fuzz](https://clusterfuzz.corp.google.com/fuzzer-stats?group_by=by-day&date_start=2020-07-11&date_end=2020-07-17&fuzzer=libFuzzer_oemcrypto_load_license_fuzz&job=libfuzzer_asan_oemcrypto)
|
|
|
|
### Issues filed by clusterfuzz - Fixing those issues
|
|
|
|
* Any issues found with the fuzz target under test are reported by clusterfuzz
|
|
[here](https://b.corp.google.com/hotlists/2442954).
|
|
|
|
* The bug will have a link to the test case that generated the bug. Download
|
|
the test case and follow the steps from
|
|
[testing fuzzer locally](#testing-fuzzer-locally) section to run the fuzzer
|
|
locally using the test case that caused the crash.
|
|
|
|
* Once the issue is fixed, consider adding the test case that caused the crash
|
|
to the seed corpus zip file. Details about seed corpus and their location
|
|
are mentioned in
|
|
[this section](#build-oemcrypto-unit-tests-to-generate-corpus).
|
|
|
|
## Corpus
|
|
|
|
* Once the fuzzer scripts are ready and running continuously using clusterfuzz
|
|
or android infrastructure, we can measure the efficiency of fuzzers by
|
|
looking at code coverage and number of new features that have been
|
|
discovered by fuzzer scripts here Fuzz script statistics.
|
|
|
|
A fuzzer which tries to start from random inputs and figure out intelligent
|
|
inputs to crash the libraries can be time consuming and not effective. A way
|
|
to make fuzzers more effective is by providing a set of valid and invalid
|
|
inputs of the library so that fuzzer can use those as a starting point.
|
|
These sets of valid and invalid inputs are called corpus.
|
|
|
|
The idea is to run OEMCrypto unit tests and read required data into binary
|
|
corpus files before calling into respective OEMCrypto APIs under test.
|
|
Writing corpus data to binary files is controlled by --generate_corpus flag.
|
|
|
|
### Build OEMCrypto unit tests to generate corpus
|
|
|
|
* Install Pre-requisites
|
|
|
|
```shell
|
|
$ sudo apt-get install gyp ninja-build
|
|
```
|
|
|
|
* download cdm source code (including ODK & OEMCrypto unit tests):
|
|
|
|
```shell
|
|
$ git clone sso://widevine-internal/cdm
|
|
```
|
|
|
|
* Build OEMCrypto unit tests and run with --generate_corpus flag to generate
|
|
corpus files:
|
|
|
|
```shell
|
|
$ cd /path/to/cdm/repo
|
|
$ export CDM_DIR=/path/to/cdm/repo
|
|
$ export PATH_TO_CDM_DIR=..
|
|
$ gyp --format=ninja --depth=$(pwd) oemcrypto/oemcrypto_unittests.gyp
|
|
$ ninja -C out/Default/
|
|
$ ./out/Default/oemcrypto_unittests --generate_corpus
|
|
```
|
|
|
|
* To avoid uploading huge binary files to git repository, the corpus files
|
|
will be saved in fuzzername_seed_corpus.zip format in blockbuster project's
|
|
oemcrypto_fuzzing_corpus GCS bucket using gsutil. If you need permissions
|
|
for blockbuster project, contact widevine-engprod@google.com.
|
|
|
|
```shell
|
|
$ gsutil cp gs://oemcrypto_fuzzing_corpus/<fuzzername_seed_corpus.zip> \
|
|
<destination_path>
|
|
```
|
|
|
|
## Testing fuzzer locally
|
|
|
|
* Corpus needed to run fuzz tests locally are available in blockbuster
|
|
project's oemcrypto_fuzzing_corpus GCS bucket. If you need permissions for
|
|
this project, contact widevine-engprod@google.com. Download corpus.
|
|
|
|
```shell
|
|
$ gsutil cp gs://oemcrypto_fuzzing_corpus/<fuzzername_seed_corpus.zip> \
|
|
<destination_path>
|
|
```
|
|
|
|
* Add flags to generate additional debugging information. Add '-g3' flag to
|
|
oemcrypto_fuzztests.gypi cflags_cc in order to generate additional debug
|
|
information locally.
|
|
|
|
* Build and test fuzz scripts locally using:
|
|
|
|
```shell
|
|
$ export CXX=clang++
|
|
$ export CC=clang
|
|
$ export GYP_DEFINES="clang=1"
|
|
$ cd /path/to/cdm/repo
|
|
$ export PATH_TO_CDM_DIR=.
|
|
$ gyp --format=ninja --depth=$(pwd) \
|
|
oemcrypto/test/fuzz_tests/oemcrypto_fuzztests.gyp
|
|
$ ninja -C out/Default/
|
|
$ mkdir /tmp/new_interesting_corpus
|
|
$ ./out/Default/fuzzer_binary /tmp/new_interesting_corpus \
|
|
/path/to/fuzz/seed/corpus/folder
|
|
```
|
|
|
|
* In order to run fuzz script against a crash input, follow the above steps
|
|
and run the fuzz binary against crash input rather than seed corpus.
|
|
|
|
```shell
|
|
$ ./out/Default/fuzzer_binary crash_input_file
|
|
```
|
|
## Adding a new OEMCrypto fuzz script
|
|
* In order to fuzz a new OEMCrypto API in future, a fuzz script can be added to
|
|
oemcrypto/test/fuzz_tests folder which ends with _fuzz.cc.
|
|
|
|
* In the program, define the function LLVMFuzzerTestOneInput with the following signature:
|
|
```
|
|
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
|
|
<your test code goes here>
|
|
return 0;
|
|
}
|
|
```
|
|
*Note*: Make sure LLVMFuzzerTestOneInput calls the function you want to fuzz.
|
|
|
|
* Add a new target to oemcrypto_fuzztests.gyp file and follow instructions in
|
|
[testing fuzzer locally](#testing-fuzzer-locally) to build and test locally.
|
|
|
|
## Generate code coverage reports locally
|
|
|
|
* Code coverage is a means of measuring fuzzer performance. We want to make
|
|
sure that our fuzzer covers all the paths in our code and make any tweeks to
|
|
fuzzer logic so we can maximize coverage to get better results.
|
|
|
|
Coverage reports for git on borg project is not automated and needs to be
|
|
generated manually. Future plan is to build a dashboard for git on borg
|
|
coverage reports.
|
|
|
|
* In order to generate coverage reports, we need to compile fuzzer binary with
|
|
flags to enable coverage. We can remove
|
|
`-fsanitize=fuzzer,address,undefined` from oemcrypto_fuzztests.gypi file as
|
|
that is needed only while fuzzing. Add following flags to both cflags_cc and
|
|
ldflags of oemcrypto_fuzztests.gypi and build fuzz binaries as mentioned in
|
|
`Testing fuzzer locally` section.
|
|
|
|
```
|
|
'-fprofile-instr-generate',
|
|
'-fcoverage-mapping',
|
|
```
|
|
|
|
* We need to run fuzzer binary against the corpus downloaded from
|
|
[clusterfuzz](https://clusterfuzz.corp.google.com/fuzzer-stats). Clock on
|
|
download link from corpus_backup column. Use gsutil command to download the
|
|
entire corpus for the fuzz binary.
|
|
|
|
* Use the following commands to generate raw profile data file with coverage
|
|
information and generate a html coverage report for a single fuzzer. More
|
|
information about clang source based coverage can be found
|
|
[here](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html). Follow
|
|
[this](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html) for steps
|
|
to combine code coverage reports of multiple fuzzers.
|
|
|
|
```shell
|
|
# Run fuzz binary against corpus backup to generate default.profraw file.
|
|
$ ./out/Default/fuzz_binary path/to/corpus/backup -runs=0
|
|
# Index raw profile files to generate coverage reports.
|
|
$ llvm-profdata merge -sparse default.profraw -o default.profdata
|
|
# Generate html coverage file.
|
|
$ llvm-cov show ./out/Default/fuzz_binary -format=html \
|
|
-instr-profile=default.profdata -o default.html
|
|
``` |