Update OEMCrypto fuzzing documentation
- Add details for triaging crashes and writing fuzz tests. - Move internal documentation not needed by partners to g3doc. - Remove infrastructure details covered in the design document. Change-Id: Ib60b2bea954f4371595b0f891434e2274366fdd2
This commit is contained in:
Ian Benz
171
libwvdrmengine/oemcrypto/test/fuzz_tests/clusterfuzz_setup.md
Normal file
171
libwvdrmengine/oemcrypto/test/fuzz_tests/clusterfuzz_setup.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# ClusterFuzz setup
|
||||
|
||||
[ClusterFuzz][1]
|
||||
|
||||
## Objective
|
||||
|
||||
* Run fuzzing on OEMCrypto public APIs on Linux by building open sourced
|
||||
ClusterFuzz source code in order to find security vulnerabilities.
|
||||
|
||||
* Partners who implement OEMCrypto can follow these instructions to build
|
||||
ClusterFuzz, the fuzzing framework and run fuzzing using fuzzer scripts
|
||||
provided by the Widevine team at Google.
|
||||
|
||||
## Glossary
|
||||
|
||||
* Fuzzing - Fuzzing is a methodology where random, interesting, unexpected
|
||||
inputs are fed to APIs in order to crash those, thereby catching any
|
||||
security vulnerabilities with the code.
|
||||
|
||||
* Fuzzing engines - [libFuzzer][4], AFL, Honggfuzz, etc. are the actual
|
||||
fuzzing engines that get the coverage information from API, use that to
|
||||
generate more interesting inputs which can be passed to fuzzer.
|
||||
|
||||
* Seed corpus - Fuzzing engine trying to generate interesting inputs from an
|
||||
empty file is not efficient. Seed corpus is the initial input that a fuzzer
|
||||
can accept and call the API with that. Fuzzing engine can then mutate this
|
||||
seed corpus to generate more inputs to fuzzer.
|
||||
|
||||
* ClusterFuzz - ClusterFuzz is a scalable fuzzing infrastructure that finds
|
||||
security and stability issues in software. Google uses ClusterFuzz to fuzz
|
||||
all Google products. ClusterFuzz provides us with the capability, tools to
|
||||
upload fuzz binaries and make use of the fuzzing engines to run fuzzing,
|
||||
find crashes and organizes the information. ClusterFuzz framework is open
|
||||
sourced, the source code can be downloaded and framework can be built
|
||||
locally or by using Google Cloud.
|
||||
|
||||
* Fuzzing output - Fuzzing is used to pass random inputs to API in order to
|
||||
ensure that API is crash resistant. We are not testing functionality via
|
||||
fuzzing. Fuzz scripts run continuously until they find a crash with the API
|
||||
under test.
|
||||
|
||||
## Build fuzz scripts
|
||||
|
||||
This section outlines the steps to build fuzz binaries that can be run
|
||||
continuously using ClusterFuzz.
|
||||
|
||||
> **Note:** All the directories mentioned below are relative to cdm repository
|
||||
> root directory.
|
||||
|
||||
1. Fuzz scripts for OEMCrypto APIs are provided by the Widevine team at Google
|
||||
located under `oemcrypto/test/fuzz_tests` directory.
|
||||
|
||||
> **Note:** Prerequisites to run the following step are [here][10]. We also
|
||||
> need to install Ninja.
|
||||
|
||||
2. Build a static library of your OEMCrypto implementation.
|
||||
* Compile and link your OEMCrypto implementation source with
|
||||
`-fsanitize=address,fuzzer` flag as per these [instructions][9] when
|
||||
building a static library.
|
||||
|
||||
* Run `./oemcrypto/test/fuzz_tests/build_partner_oemcrypto_fuzztests
|
||||
<oemcrypto_static_library_path>` script from cdm repository root
|
||||
directory.
|
||||
|
||||
* This will generate fuzz binaries under the `out/Default` directory.
|
||||
|
||||
> **Note:** Alternatively, you can use your own build systems, for which you
|
||||
> will need to define your own build files with the OEMCrypto fuzz source
|
||||
> files included. You can find the the fuzz source files in
|
||||
> `oemcrypto/test/fuzz_tests/partner_oemcrypto_fuzztests.gyp` and
|
||||
> `oemcrypto/test/fuzz_tests/partner_oemcrypto_fuzztests.gypi`.
|
||||
|
||||
3. Seed corpus for each fuzz script can be found under
|
||||
`oemcrypto/test/fuzz_tests/corpus` directory. Some fuzzers are simple and do
|
||||
not have seed corpus associated with them.
|
||||
|
||||
4. Create a zip file `oemcrypto_fuzzers_yyyymmddhhmmss.zip` with fuzz binaries
|
||||
and respective seed corpus zip files. Structure of a sample zip file with
|
||||
fuzzer binaries and seed corpus would look like this:
|
||||
|
||||
```
|
||||
* fuzzerA
|
||||
* fuzzerA_seed_corpus.zip
|
||||
* fuzzerB
|
||||
* fuzzerB_seed_corpus.zip
|
||||
* fuzzerC (fuzzerC doesn't have seed corpus associated with it)
|
||||
```
|
||||
|
||||
## Build ClusterFuzz
|
||||
|
||||
OEMCrypto implementation can be fuzzed by building ClusterFuzz code, which is
|
||||
open source, and using it to run fuzzing. Use a Linux VM to build ClusterFuzz.
|
||||
|
||||
> **Note:** You may see some issues with Python modules missing. Please install
|
||||
> those modules if you see errors. If you have multiple versions of Python on
|
||||
> the VM, then use `python<version> -m pipenv shell` when you are at [this][3]
|
||||
> step.
|
||||
|
||||
Follow these [instructions][2] in order to download the ClusterFuzz repository,
|
||||
build it locally or create a continuous fuzz infrastructure setup using Google
|
||||
Cloud.
|
||||
|
||||
## Run fuzzers on local ClusterFuzz instance
|
||||
|
||||
If you prefer to run fuzzing on a local machine instead of having a production
|
||||
setup using Google Cloud, then follow these [instructions][5] to add a job to
|
||||
the local ClusterFuzz instance.
|
||||
|
||||
> **Note:** Job name should have a fuzzing engine and sanitizer as part of it. A
|
||||
> libFuzzer and AddressSanitizer job should have libfuzzer_asan in the job name.
|
||||
|
||||
* Create a job e:g:`libfuzzer_asan_oemcrypto` and upload previously created
|
||||
`oemcrypto_fuzzers_yyyymmddhhmmss.zip` as a custom build. Future uploads of
|
||||
zip file should have a name greater than current name. Following the above
|
||||
naming standard will ensure zip file names are always in ascending order.
|
||||
|
||||
* Once the job is added and ClusterFuzz bot is running, fuzzing should be up
|
||||
and running. Results can be monitored as mentioned [here][6].
|
||||
|
||||
* On a local ClusterFuzz instance, only one fuzzer is being fuzzed at a time.
|
||||
|
||||
> **Note:** Fuzzing is time consuming. Finding issues as well as ClusterFuzz
|
||||
> regressing and fixing the issues can take time. We need fuzzing to run at
|
||||
> least for a couple of weeks to have good coverage.
|
||||
|
||||
## Find fuzz crashes
|
||||
|
||||
* Once the ClusterFuzz finds an issue, it logs crash information such as the
|
||||
build, test case and stack trace for the crash.
|
||||
|
||||
* Test cases tab should show the fuzz crash and test case that caused the
|
||||
crash. Run `./fuzz_binary <test_case>` in order to debug the crash locally.
|
||||
|
||||
More information about different types of logs is below:
|
||||
|
||||
* [Bot logs][7] will show information related to fuzzing, number of crashes
|
||||
that a particular fuzzer finds, number of new crashes, number of known
|
||||
crashes etc.
|
||||
|
||||
* [Local GCS][8] in your ClusterFuzz checkout folder will store the fuzz
|
||||
binaries that are being fuzzed, seed corpus etc.
|
||||
|
||||
* `local_gcs/test-fuzz-logs-bucket` will store information related to fuzz
|
||||
crashes if any were found by the fuzzing engine. It will store crash
|
||||
information categorized by fuzzer and by each day. It will also store test
|
||||
case that caused the crash.
|
||||
|
||||
* `/path/to/my-bot/clusterfuzz/log.txt` will have any log information from
|
||||
fuzzer script and OEMCrypto implementation.
|
||||
|
||||
## Fix issues
|
||||
|
||||
1. Once you are able to debug using the crash test case, apply fix to the
|
||||
implementation, create `oemcrypto_fuzzers_yyyymmddhhmmss.zip` with latest
|
||||
fuzz binaries.
|
||||
|
||||
2. Upload the latest fuzz binary to the fuzz job that was created earlier.
|
||||
Fuzzer will recognize the fix and mark the crash as fixed in test cases tab
|
||||
once the regression finishes. You do not need to update crashes as fixed,
|
||||
ClusterFuzz will do that.
|
||||
|
||||
[1]: https://google.github.io/clusterfuzz/
|
||||
[2]: https://google.github.io/clusterfuzz/getting-started/
|
||||
[3]: https://google.github.io/clusterfuzz/getting-started/prerequisites/#loading-pipenv
|
||||
[4]: https://llvm.org/docs/LibFuzzer.html
|
||||
[5]: https://google.github.io/clusterfuzz/setting-up-fuzzing/libfuzzer-and-afl/
|
||||
[6]: https://google.github.io/clusterfuzz/setting-up-fuzzing/libfuzzer-and-afl/#checking-results
|
||||
[7]: https://google.github.io/clusterfuzz/getting-started/local-instance/#viewing-logs
|
||||
[8]: https://google.github.io/clusterfuzz/getting-started/local-instance/#local-google-cloud-storage
|
||||
[9]: https://google.github.io/clusterfuzz/setting-up-fuzzing/libfuzzer-and-afl/#libfuzzer
|
||||
[10]: https://google.github.io/clusterfuzz/setting-up-fuzzing/libfuzzer-and-afl/#prerequisites
|
||||
Reference in New Issue
Block a user