widevine-partner/ce_cdm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Source release 16.4.0

Browse Source
This commit is contained in:
John W. Bruce
2020-10-09 16:08:56 -07:00
parent 160df9f57a
commit 9d17a531ee
562 changed files with 52913 additions and 37426 deletions
Download Patch File Download Diff File Expand all files Collapse all files

120
oemcrypto/test/fuzz_tests/README.md
View File

@@ -139,9 +139,76 @@
```
*Note*: Make sure LLVMFuzzerTestOneInput calls the function you want to fuzz.
* Add a new target to oemcrypto_fuzztests.gyp file and follow instructions in
* Add a new target to oemcrypto_fuzztests.gyp file and follow instructions in
[testing fuzzer locally](#testing-fuzzer-locally) to build and test locally.
## Building OEMCrypto fuzz scripts and uploading them to Google Cloud Storage:
* We are using Google Cloud Buid (GCB) in order to setup continuous
integration which uploads OEMCrypto fuzz binaries to Google Cloud Storage.
GCB expects build script in form of a docker image that is uploaded to
Google Container Registry(GCR).
The cloud build scripts (docker images) for widevine projects are
[here](https://widevine-internal.googlesource.com/cloud/+/refs/heads/master/docker/README.md)
Refer to README of the project to setup a new docker image and uploading
the image to GCR.
* Git on borg repository needs to be integrated with GCB and a git trigger
needs to be set up in order to achieve continuous integration. Git trigger
will mention which docker image the GCB needs to use in order to build fuzz
binaries. GCB searches for docker images from GCR.
Design document lists the steps to create a git trigger.
### Adding a new fuzz script to the build script:
* In order to update build script such as adding a new fuzzer to build script,
we need to update the build script in docker image from cloud repository.
[Build script.](https://widevine-internal.googlesource.com/cloud/+/refs/heads/master/docker
/cloud_build/oemcrypto/release/ubuntu/fuzz/build.sh)
Add the new fuzz script name to fuzzers variable and follow steps in README
to upload new docker image. Make sure you update the tag to be higher than
latest version in GCR.
Run the following command from your machine to update the docker image tag
in the git trigger.
```shell
stubby call --rpc_creds_file=/tmp/mint.txt \
blade:alphasource-ci-proctor-metadata-service-prod \
ProctorMetadataService.UpdateTrigger --proto2 <<EOF
trigger {
cloud_project_number: 257246079067
name: "cdm-git-trigger"
id: "e8939c9a-d971-4c05-91b5-e0544abf872b"
state: LIVE
git_trigger {
url: "https://widevine-internal.googlesource.com/cdm"
branch_name: "master"
}
build_configs {
build {
steps {
name: "gcr.io/google.com/blockbuster-1154/
cloud-build-oemcrypto-release-ubuntu-fuzz:LATEST_TAG_VERSION"
}
}
}
result_config {
email_config {
notify_condition {
condition: ON_FAILURE
}
to_address: "wideving-engprod@google.com"
}
}
}
EOF
```
## Generate code coverage reports locally
* Code coverage is a means of measuring fuzzer performance. We want to make
@@ -152,36 +219,29 @@
generated manually. Future plan is to build a dashboard for git on borg
coverage reports.
* In order to generate coverage reports, we need to compile fuzzer binary with
flags to enable coverage. We can remove
`-fsanitize=fuzzer,address,undefined` from oemcrypto_fuzztests.gypi file as
that is needed only while fuzzing. Add following flags to both cflags_cc and
ldflags of oemcrypto_fuzztests.gypi and build fuzz binaries as mentioned in
`Testing fuzzer locally` section.
### Generate code coverage reports using script from Google cloud build
* A docker image with script to generate code coverage reports for oemcrypto
fuzz scripts is linked with a GCB trigger
`oemcrypto-fuzzing-code-coverage-git-trigger`. More information about clang
source based coverage can be found
[here](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html).
```
'-fprofile-instr-generate',
'-fcoverage-mapping',
```
* This trigger when invoked compiles oemcrypto fuzz scripts with clang source
based code coverage enabled, downloads latest corpus from cluster fuzz
for the respective fuzzer, generates and uploads code coverage html reports
to [GCS](https://pantheon.corp.google.com/storage/browser/oemcrypto_fuzzing_code_coverage_reports;tab=objects?forceOnBucketsSortingFiltering=false&project=google.com:blockbuster-1154&prefix=).
* We need to run fuzzer binary against the corpus downloaded from
[clusterfuzz](https://clusterfuzz.corp.google.com/fuzzer-stats). Clock on
download link from corpus_backup column. Use gsutil command to download the
entire corpus for the fuzz binary.
* The trigger can be invoked manually using cloud scheduler
`oemcrypto_fuzzing_code_coverage_reports`.
* Use the following commands to generate raw profile data file with coverage
information and generate a html coverage report for a single fuzzer. More
information about clang source based coverage can be found
[here](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html). Follow
[this](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html) for steps
to combine code coverage reports of multiple fuzzers.
* In order to generate latest code coverage reports from master branch,
go to pantheon->cloud scheduler->oemcrypto_fuzzing_code_coverage_reports and
click on `RUN NOW` button.
```shell
# Run fuzz binary against corpus backup to generate default.profraw file.
$ ./out/Default/fuzz_binary path/to/corpus/backup -runs=0
# Index raw profile files to generate coverage reports.
$ llvm-profdata merge -sparse default.profraw -o default.profdata
# Generate html coverage file.
$ llvm-cov show ./out/Default/fuzz_binary -format=html \
-instr-profile=default.profdata -o default.html
```
* The above step should invoke a google cloud build. Go to cloud build console
and find latest build job with Trigger Name
`oemcrypto-fuzzing-code-coverage-git-trigger`.
* Once the build job is successful, latest code coverage reports can be
downloaded from [GCS](https://pantheon.corp.google.com/storage/browser/oemcrypto_fuzzing_code_coverage_reports;tab=objects?forceOnBucketsSortingFiltering=false&project=google.com:blockbuster-1154&prefix=).
The coverage report folder uploaded to GCS is appended with timestamp.

182
oemcrypto/test/fuzz_tests/oemcrypto_decrypt_cenc_fuzz.cc Normal file
View File

@@ -0,0 +1,182 @@
// Copyright 2020 Google LLC. All Rights Reserved. This file and proprietary
// source code may only be used and distributed under the Widevine Master
// License Agreement.
#include "FuzzedDataProvider.h"
#include "OEMCryptoCENC.h"
#include "log.h"
#include "oemcrypto_fuzz_helper.h"
#include "oemcrypto_fuzz_structs.h"
namespace wvoec {
const size_t MAX_FUZZ_SAMPLE_SIZE = 5 * MB;
// Free dynamic memory allocated by fuzzer script.
void FreeOutputBuffers(OEMCrypto_SESSION session_id,
OEMCrypto_SampleDescription* sample_description,
size_t sample_index, int* secure_fd_array) {
for (size_t i = 0; i < sample_index; i++) {
OEMCrypto_DestBufferDesc fuzzed_output_descriptor =
sample_description[i].buffers.output_descriptor;
switch (fuzzed_output_descriptor.type) {
case OEMCrypto_BufferType_Clear: {
delete[] fuzzed_output_descriptor.buffer.clear.address;
break;
}
case OEMCrypto_BufferType_Secure: {
OEMCrypto_FreeSecureBuffer(session_id, &fuzzed_output_descriptor,
secure_fd_array[i]);
break;
}
case OEMCrypto_BufferType_Direct: {
break;
}
}
}
}
// Function to initialize output buffer pointers by allocating memory.
// Limiting output buffer size to 5 MB as 4 MB is maximum size specified
// by resource rating tier documentation.
bool InitializeOutputBuffers(OEMCrypto_SESSION session_id,
OEMCrypto_DestBufferDesc& output_descriptor,
size_t sample_index,
vector<int>& secure_fd_array) {
switch (output_descriptor.type) {
case OEMCrypto_BufferType_Clear: {
output_descriptor.buffer.clear
.address = new OEMCrypto_SharedMemory[std::min(
MAX_FUZZ_SAMPLE_SIZE, output_descriptor.buffer.clear.address_length)];
return true;
}
case OEMCrypto_BufferType_Secure: {
int* secure_fd;
OEMCryptoResult sts = OEMCrypto_AllocateSecureBuffer(
session_id,
std::min(MAX_FUZZ_SAMPLE_SIZE,
output_descriptor.buffer.secure.handle_length),
&output_descriptor, secure_fd);
if (sts == OEMCrypto_SUCCESS) secure_fd_array[sample_index] = *secure_fd;
return sts == OEMCrypto_SUCCESS;
}
case OEMCrypto_BufferType_Direct: {
return true;
}
}
}
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
// Redirect printf and log statements from oemcrypto functions to a file to
// reduce noise
RedirectStdoutToFile();
size_t samples_length;
// Split data using separator.
auto inputs = SplitInput(data, size);
if (inputs.size() < 2) {
return 0;
}
OEMCrypto_Decrypt_Cenc_Fuzz fuzzed_structure;
if (inputs[0].size() < sizeof(fuzzed_structure)) {
return 0;
}
// Copy OEMCrypto_Decrypt_Cenc_Fuzz from input data.
memcpy(&fuzzed_structure, data, sizeof(fuzzed_structure));
ConvertDataToValidEnum(OEMCrypto_CipherMode_MaxValue,
&fuzzed_structure.cipher_mode);
size_t remaining_size_for_samples =
inputs[0].size() - sizeof(fuzzed_structure);
// Initialize FDP structures to read data using inbuilt functions.
FuzzedDataProvider fuzzed_sample_data(data + sizeof(fuzzed_structure),
remaining_size_for_samples);
FuzzedDataProvider fuzzed_subsample_data(inputs[1].data(), inputs[1].size());
// Read subsamples from fuzzed data.
vector<OEMCrypto_SubSampleDescription> subsamples;
while (fuzzed_subsample_data.remaining_bytes() >
sizeof(OEMCrypto_SubSampleDescription)) {
OEMCrypto_SubSampleDescription subsample;
fuzzed_subsample_data.ConsumeData(&subsample,
sizeof(OEMCrypto_SubSampleDescription));
subsamples.push_back(subsample);
}
if (subsamples.size() == 0) {
return 0;
}
// Infer samples_length from fuzzed data.
size_t sample_description_size = sizeof(OEMCrypto_SampleDescription);
samples_length =
fuzzed_sample_data.remaining_bytes() / sample_description_size;
if (samples_length == 0) {
return 0;
}
// Initialize sample_descriptions array.
vector<OEMCrypto_SampleDescription> sample_descriptions(samples_length);
// Create array to maintain secure_fd buffer values for secure buffers.
vector<int> secure_fd_array(samples_length);
OEMCryptoLicenseAPIFuzz license_api_fuzz;
Session* session = license_api_fuzz.session();
// Copy samples from fuzzed data.
size_t input_subsample_index = 0;
size_t total_input_data_length = 0;
for (size_t i = 0; i < samples_length; i++) {
fuzzed_sample_data.ConsumeData(&sample_descriptions[i],
sample_description_size);
ConvertDataToValidEnum(
OEMCrypto_BufferType_MaxValue,
&sample_descriptions[i].buffers.output_descriptor.type);
// Copy random data into input sample data. Cap input data length at 5 MB,
// 1 MB higher than that described by resource rating tier.
total_input_data_length += std::min(
MAX_FUZZ_SAMPLE_SIZE, sample_descriptions[i].buffers.input_data_length);
// Copy sub sample data.
sample_descriptions[i].subsamples = &subsamples[input_subsample_index];
input_subsample_index += sample_descriptions[i].subsamples_length;
if (input_subsample_index > subsamples.size()) return 0;
} // Sample loop.
// Allocate input/output buffers for each sample description.
vector<OEMCrypto_SharedMemory> input_buffer(total_input_data_length);
RAND_bytes(input_buffer.data(), total_input_data_length);
size_t input_buffer_index = 0;
for (size_t i = 0; i < samples_length; i++) {
sample_descriptions[i].buffers.input_data =
&input_buffer[input_buffer_index];
input_buffer_index += std::min(
MAX_FUZZ_SAMPLE_SIZE, sample_descriptions[i].buffers.input_data_length);
// Create output buffer pointers. If secure buffer is not supported, we
// explicitly convert to clear buffer and fuzz.
if (!InitializeOutputBuffers(
session->session_id(),
sample_descriptions[i].buffers.output_descriptor, i,
secure_fd_array)) {
LOGI(
"[OEMCrypto decrypt CENC fuzz] Secure buffers are not supported. Use "
"clear buffer instead.");
sample_descriptions[i].buffers.output_descriptor.type =
OEMCrypto_BufferType_Clear;
InitializeOutputBuffers(session->session_id(),
sample_descriptions[i].buffers.output_descriptor,
i, secure_fd_array);
}
}
// Load license and call decrypt_cenc API.
license_api_fuzz.LoadLicense();
OEMCrypto_SelectKey(session->session_id(), session->license().keys[0].key_id,
session->license().keys[0].key_id_length,
fuzzed_structure.cipher_mode);
OEMCrypto_DecryptCENC(session->session_id(), sample_descriptions.data(),
samples_length, &fuzzed_structure.pattern);
FreeOutputBuffers(session->session_id(), sample_descriptions.data(),
samples_length, secure_fd_array.data());
return 0;
}
} // namespace wvoec

17
oemcrypto/test/fuzz_tests/oemcrypto_fuzz_helper.cc
View File

@@ -5,4 +5,21 @@
namespace wvoec {
void RedirectStdoutToFile() { freopen("log.txt", "a", stdout); }
std::vector<std::vector<uint8_t>> SplitInput(const uint8_t* data, size_t size) {
std::vector<std::vector<uint8_t>> result;
auto current_pos = data;
auto end = data + size;
// Using memmem to find separator
while (const uint8_t* pos = reinterpret_cast<const uint8_t*>(
memmem(current_pos, end - current_pos, kFuzzDataSeparator,
sizeof(kFuzzDataSeparator)))) {
result.push_back({current_pos, pos});
current_pos = pos + sizeof(kFuzzDataSeparator);
}
if (current_pos < end) {
result.push_back({current_pos, end});
}
return result;
}
} // namespace wvoec

14
oemcrypto/test/fuzz_tests/oemcrypto_fuzz_helper.h
View File

@@ -9,9 +9,9 @@
#include "FuzzedDataProvider.h"
#include "OEMCryptoCENC.h"
#include "oec_device_features.h"
#include "oec_session_util.h"
#include "oemcrypto_corpus_generator_helper.h"
#include "oemcrypto_session_tests_helper.h"
namespace wvoec {
// Initial setup to create a valid OEMCrypto state such as initializing crypto
// firmware/hardware, installing golden key box etc. in order to fuzz
@@ -42,6 +42,15 @@ class OEMCryptoLicenseAPIFuzz : public InitializeFuzz {
LicenseRoundTrip& license_messages() { return license_messages_; }
Session* session() { return &session_; }
void LoadLicense() {
license_messages_.SignAndVerifyRequest();
license_messages_.CreateDefaultResponse();
license_messages_.EncryptAndSignResponse();
license_messages_.LoadResponse();
}
private:
Session session_;
LicenseRoundTrip license_messages_;
@@ -89,6 +98,9 @@ void ConvertDataToValidEnum(T max_enum_value, T* t) {
// Redirect printf and log statements from oemcrypto functions to a file to
// reduce noise
void RedirectStdoutToFile();
// Function to split fuzzer input using delimiter "-_^_".
std::vector<std::vector<uint8_t>> SplitInput(const uint8_t* data, size_t size);
} // namespace wvoec
#endif // OEMCRYPTO_FUZZ_HELPER_H_

11
oemcrypto/test/fuzz_tests/oemcrypto_fuzz_structs.h
View File

@@ -25,6 +25,17 @@ struct OEMCrypto_Request_Fuzz {
// that ODK parses and actual message buffer to the request APIs.
size_t signature_length;
size_t core_message_length;
// Request message is of variable length and not included in this structure.
};
struct OEMCrypto_Decrypt_Cenc_Fuzz {
// Corpus format is as below, let | be separator.
// cipher_mode + pattern + sample_data for all samples |
// subsample_data for all samples
OEMCryptoCipherMode cipher_mode;
OEMCrypto_CENCEncryptPatternDesc pattern;
// Sample data and subsample data are of variable length and not included in
// this structure.
};
} // namespace wvoec

12
oemcrypto/test/fuzz_tests/oemcrypto_fuzztests.gyp
View File

@@ -48,5 +48,17 @@
'oemcrypto_renewal_request_fuzz.cc',
],
},
{
'target_name': 'oemcrypto_decrypt_cenc_fuzz',
'sources': [
'oemcrypto_decrypt_cenc_fuzz.cc',
],
},
{
'target_name': 'oemcrypto_load_entitled_content_keys_fuzz',
'sources': [
'oemcrypto_load_entitled_content_keys_fuzz.cc',
],
},
],
}

14
oemcrypto/test/fuzz_tests/oemcrypto_fuzztests.gypi
View File

@@ -62,6 +62,10 @@
# Need -g flag to include source line numbers in error stack trace.
'-g',
],
'ldflags': [
'-fPIC',
'-fsanitize=fuzzer,address,undefined',
],
}],
['generate_code_coverage_report=="true"', {
# Include flags to build fuzzer binaries to generate source based code coverage reports.
@@ -70,12 +74,14 @@
'-fprofile-instr-generate',
'-fcoverage-mapping',
],
'ldflags': [
'-fPIC',
'-fsanitize=fuzzer,address,undefined',
'-fprofile-instr-generate',
'-fcoverage-mapping',
],
}],
], # conditions
'ldflags': [
'-fPIC',
'-fsanitize=fuzzer,address,undefined',
],
'libraries': [
'-lpthread',
],

66
oemcrypto/test/fuzz_tests/oemcrypto_load_entitled_content_keys_fuzz.cc Normal file
View File

@@ -0,0 +1,66 @@
// Copyright 2020 Google LLC. All Rights Reserved. This file and proprietary
// source code may only be used and distributed under the Widevine Master
// License Agreement.
#include "FuzzedDataProvider.h"
#include "oemcrypto_fuzz_helper.h"
#include "oemcrypto_fuzz_structs.h"
namespace wvoec {
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
// Redirect printf and log statements from oemcrypto functions to a file to
// reduce noise
RedirectStdoutToFile();
// Corpus format is as below, let | be separator.
// message buffer with key data | entitled content key object array with
// offsets and lengths to read key data from message buffer.
// Split data using separator.
auto inputs = SplitInput(data, size);
if (inputs.size() < 2) {
return 0;
}
FuzzedDataProvider fuzzed_entitled_content_key_array(inputs[1].data(),
inputs[1].size());
// Message to be verified. Return 0 if key data buffer is empty.
if (inputs[0].size() == 0) {
return 0;
}
// Copy data to OEMCrypto_EntitledContentKeyObject array.
size_t entitled_content_key_object_size =
sizeof(OEMCrypto_EntitledContentKeyObject);
size_t entitled_content_key_array_length =
fuzzed_entitled_content_key_array.remaining_bytes() /
entitled_content_key_object_size;
if (entitled_content_key_array_length == 0) {
return 0;
}
OEMCrypto_EntitledContentKeyObject* entitled_content_key_array =
new OEMCrypto_EntitledContentKeyObject[entitled_content_key_array_length];
for (size_t i = 0; i < entitled_content_key_array_length; i++) {
fuzzed_entitled_content_key_array.ConsumeData(
&entitled_content_key_array[i], entitled_content_key_object_size);
}
OEMCryptoLicenseAPIFuzz license_api_fuzz;
// Setting up state. Load default entitlement license to load entitlement
// keys into sessions key table.
license_api_fuzz.license_messages().set_license_type(
OEMCrypto_EntitlementLicense);
license_api_fuzz.LoadLicense();
// Call OEMCrypto_LoadEntitledContentKeys with fuzzed buffers.
Session* session = license_api_fuzz.session();
uint8_t* fuzzed_key_data = inputs[0].data();
size_t fuzzed_key_data_size = inputs[0].size();
OEMCrypto_LoadEntitledContentKeys(
session->session_id(), fuzzed_key_data, fuzzed_key_data_size,
entitled_content_key_array_length, entitled_content_key_array);
delete[] entitled_content_key_array;
return 0;
}
} // namespace wvoec

14
oemcrypto/test/fuzz_tests/oemcrypto_load_renewal_fuzz.cc
View File

@@ -23,16 +23,16 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
const size_t renewal_response_size =
size - sizeof(OEMCrypto_Renewal_Response_Fuzz);
OEMCryptoLicenseAPIFuzz license_api_fuzz;
license_api_fuzz.license_messages().SignAndVerifyRequest();
license_api_fuzz.license_messages().CreateDefaultResponse();
OEMCryptoRenewalAPIFuzz renewal_response_fuzz;
renewal_response_fuzz.license_messages().SignAndVerifyRequest();
renewal_response_fuzz.license_messages().CreateDefaultResponse();
// Inject timer limits from fuzzed input to timer_limits field from
// core license response.
license_api_fuzz.license_messages().InjectFuzzedTimerLimits(fuzzed_data);
license_api_fuzz.license_messages().EncryptAndSignResponse();
license_api_fuzz.license_messages().LoadResponse();
renewal_response_fuzz.license_messages().InjectFuzzedTimerLimits(fuzzed_data);
renewal_response_fuzz.license_messages().EncryptAndSignResponse();
renewal_response_fuzz.license_messages().LoadResponse();
OEMCryptoRenewalAPIFuzz renewal_response_fuzz;
// Call renewal response API using fuzzed data.
renewal_response_fuzz.renewal_messages().SignAndVerifyRequest();
renewal_response_fuzz.renewal_messages().InjectFuzzedResponseData(
fuzzed_data, renewal_response, renewal_response_size);