Merges to android Pi release (part 9)

These are a set of CLs merged from the wv cdm repo to the android repo.

* Make Android NDK Builds Work With Latest BoringSSL

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/37000 ]

  The latest updates to BoringSSL require C99 or later. Our NDK-based
  builds (OEMCrypto Variants & Fastball) were not specifying a C standard.
  This patch adds compiler flags so that C files are compiled as C11 now.
  Note that this is about the *C* standard in use, not the *C++* standard,
  which this patch leaves untouched.

  BUG: 67907873
  Test: build_android_mock.sh

*  Update BoringSSL to f7412cb072cc6b1847140e0c4f8b3ceeccd0e708

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/36761 ]

  This is the result of running UPDATE_BORINGSSL.sh. Future runs of this
  script should produce much smaller sets of changed files, but because
  the BoringSSL revision already in this directory was so old and
  contained many extraneous files from the Android operating system, the
  set of changed files is extensive this time.

  BUG: 67907873

* Refactoring the build files.

  Author: Vasantha Rao Polipelli <vasanthap@google.com>

  [ Merge of http://go/wvgerrit/37041 ]

  Move all common build dependencies to .gypi so that all fuzz test
  binary targets can be added to .gyp file without repeating code.

* Introduce service certificate request property

  Author: Rahul Frias <rfrias@google.com>

  [ Merge of http://go/wvgerrit/36941 ]

  Platforms differ on whether they allows service certificates to be
  requested if privacy mode is enabled and a certificate is not present.
  This property allows behavior to be configurable.

  Generating the service certificate request will be introduced
  in a follow on CL.

  BUG: 68328352

* Deprecate using keyboxes as identification

  Author: Rahul Frias <rfrias@google.com>

  [ Merge of http://go/wvgerrit/36740 ]

  Previously some platforms supported using keyboxes rather than
  certificates as the identification tokens in the license request
  message. All platforms that share core CDM code of the master branch now
  either provision using a keybox and use a DRM certificate or an
  OEM certificate as identification. No future usage of keyboxes
  as identifying tokens is planned.

  Since the platform property use_certificates_as_identification
  is always set to true, the negative code paths are never taken and
  can be removed.

* OEMCrypto_GenerateSignature API Fuzz Test.

  Author: Vasantha Rao Polipelli <vasanthap@google.com>

  [ Merge of http://go/wvgerrit/36863 ]

  - The first automated API fuzz test.
  - Also sumitting the corpus for the API fuzzed.

* Add Script to Update BoringSSL from Source

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/36760 ]

  Adds a script to third_party/boringssl/ that, when run, deletes all the
  auto-generated files in the generated/ directory and regenerates them
  from scratch, starting from the latest public HEAD of BoringSSL.

  Bug: 67907873

* Fix Fastball / OEMCrypto Variant BoringSSL Makefiles

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/36926 ]

  Previously, when moving the BoringSSL source within the tree, I was not
  able to verify that I had not broken the NDK-compatible makefiles used
  by Fastball because that build is broken on master. I had to make a
  best-guess as to how they should be updated and hope.

  Now, however, I have been informed that the OEMCrypto Variants also use
  these makefiles, and I have been able to use that build to find where I
  broke them and get them fully working.

  Bug: 67386164
  Test: build_android_mock.sh

* Add kit/ to BoringSSL Include Path for Fastball & OEMCrypto Variants

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/36925 ]

  When I moved the BoringSSL source in the tree, I updated the Android.mk
  files that pointed to it in order to build it. I did not realize that
  some makefiles outside that directory also contained hardcoded pointers
  into that directory. These references broke after the move. This patch
  fixes those paths to point to the new BoringSSL location.

  Bug: 67386164
  Test: build_android_mock.sh

* OEMCrypto Unit Test Refactor.

  Author: Vasantha Rao Polipelli <vasanthap@google.com>

  [ Merge of http://go/wvgerrit/36562 ]

  Refactoring OEMCrypto Tests so the Session Utility test code can be reused in fuzz tests.

* Reorder license server config table to match ids

  Author: Jeff Fore <jfore@google.com>

  [ Merge of http://go/wvgerrit/36743 ]

* Separate Hand-Written BoringSSL Files from Downloaded/Generated Ones

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/36561 ]

  I want to make updating BoringSSL as simple as possible for us going
  forward. A future commit will add a script that automatically downloads
  and sets up the latest version of BoringSSL. To facilitate this script,
  a clear distinction needs to be made between the files that can be
  downloaded with / regenerated from the BoringSSL source and the files
  that are maintained by us by hand.

  The version of BoringSSL in this change is exactly the same as the one
  already in this directory. It has just been moved one folder deeper.

  Bug: 67907873

* Remove BoringSSL Symlinks, They Are Confusing Gerrit

  Author: John W. Bruce <juce@google.com>

  [ Merge of http://go/wvgerrit/36560 ]

  There are some symlinks in the current copy of BoringSSL that are
  causing headaches when I try to upload future changes to Gerrit. These
  were inherited from the Android OS and are not used by our build
  anywhere. They would be wiped out when I update BoringSSL anyway, but
  wiping them out in a separate change before I upload any other changes
  avoids confusing Gerrit.

  Bug: 67907873

* Add group master key id to support sublicense master
  key rotation, and content identification.

  Author: Jeff Fore <jfore@google.com>

  [ Merge of http://go/wvgerrit/36180 ]

* OEMCrypto Fuzzer test framework

  Author: Vasantha Rao Polipelli <vasanthap@google.com>

  [ Merge of http://go/wvgerrit/36280 ]

  - Adding a sample fuzz test.
  - Adding build scripts for building the new Fuzz Tests to come.

  Design doc: go/oemcrypt_ref_impl_fuzz

* Build Mod Mock with C++ 11

  Author: Fred Gylys-Colwell <fredgc@google.com>

  [ Merge of http://go/wvgerrit/36328 ]

  This should fix the android oemcrypto mock build:
  http://go/wvbuild/job/Android_OEMCrypto_Variants

BUG: 71650075
Test: Not currently passing. Will be addressed in a subsequent
    commit in the chain.

Change-Id: Ic4d5be3118ef97e3f7d386149a2b5d9be8f0a87e
This commit is contained in:
Rahul Frias
2018-01-10 18:01:48 -08:00
parent 1884cf738e
commit 81d607c008
35 changed files with 688 additions and 1055 deletions

View File

@@ -6,18 +6,13 @@
#define WVCDM_METRICS_EVENT_METRIC_H_
#include <cstdarg>
#include <functional>
#include <map>
#include <memory>
#include <ostream>
#include <sstream>
#include <stdint.h>
#include <string>
#include <vector>
#include "lock.h"
#include "distribution.h"
#include "field_tuples.h"
#include "lock.h"
#include "metric_serialization.h"
namespace wvcdm {
@@ -60,14 +55,6 @@ class BaseEventMetric : public MetricSerializable {
Lock internal_lock_;
};
// This is a placeholder type for unused type parameters.
struct Unused {
// Required for compilation. Should never be used.
inline friend std::ostream& operator<< (std::ostream& out, const Unused&)
{ return out; }
};
// This class converts the size_t value into the highest power of two
// below the value. E.g. for 7, the value is 4. For 11, the value is 8.
// This class is intended to simplify the use of EventMetric Fields that may
@@ -113,7 +100,7 @@ class Pow2Bucket {
// on certain "field" values. For example, if a particular operation can run in
// one of two modes, it's useful to track the latency of the operation in each
// mode separately. You can use Fields to define how to breakdown the
// statistics. Each Field is a separate dimention. The statistics for each
// statistics. Each Field is a separate dimension. The statistics for each
// combination of field values are tracked independently.
//
// Example usage:
@@ -123,19 +110,22 @@ class Pow2Bucket {
//
// my_metric.Record(1, 7, 23); // (latency value, request type, error code).
//
// A MetricNotification may be used to allow the EventMetric instance to
// notify that an update occurred and provide the updated value.
// The EventMetric supports serialization. A call to Serialize will
// serialize all values to the provided MetricsSerializer instance.
//
// class MyMetricNotification : public MetricNotification {
// example:
//
// class MyMetricSerializer : public MetricSerializer {
// // Add implementation here.
// }
//
// MyMetricNotification notification;
// my_metric.Publish(notification);
template<typename F1=Unused,
typename F2=Unused,
typename F3=Unused,
typename F4=Unused>
// MyMetricSerializer serializer;
// my_metric.Serialize(&serializer);
//
template<typename F1=util::Unused,
typename F2=util::Unused,
typename F3=util::Unused,
typename F4=util::Unused>
class EventMetric : public BaseEventMetric {
public:
// Create an EventMetric instance with the name |metric_name|.
@@ -162,116 +152,16 @@ class EventMetric : public BaseEventMetric {
// Record will update the statistics of the EventMetric broken down by the
// given field values.
void Record(double value,
F1 field1 = Unused(),
F2 field2 = Unused(),
F3 field3 = Unused(),
F4 field4 = Unused());
F1 field1 = util::Unused(),
F2 field2 = util::Unused(),
F3 field3 = util::Unused(),
F4 field4 = util::Unused());
private:
friend class EventMetricTest;
std::vector<std::string> field_names_;
};
// This is an internal namespace for helper functions only.
namespace impl {
// This method formats the collection of field name/value pairs.
// The format of the string is:
//
// [{field:value[&field:value]*}]
//
// If there are no pairs, returns a blank string.
//
// TODO(blueeyes): Add a check for the count of field_names and the count
// of field values (check that the fields are not type Unused).
template<typename F1, typename F2, typename F3, typename F4>
std::string MakeFieldNameString(const std::vector<std::string>& field_names,
const F1 field1, const F2 field2,
const F3 field3, const F4 field4) {
std::stringstream field_name_and_values;
std::vector<std::string>::const_iterator field_name_iterator =
field_names.begin();
if (field_name_iterator == field_names.end()) {
return field_name_and_values.str();
}
// There is at least one name/value pair. prepend open brace.
field_name_and_values << "{";
field_name_and_values << *field_name_iterator << ':' << field1;
if (++field_name_iterator == field_names.end()) {
field_name_and_values << "}";
return field_name_and_values.str();
}
field_name_and_values << '&' << *field_name_iterator << ':' << field2;
if (++field_name_iterator == field_names.end()) {
field_name_and_values << "}";
return field_name_and_values.str();
}
field_name_and_values << '&' << *field_name_iterator << ':' << field3;
if (++field_name_iterator == field_names.end()) {
field_name_and_values << "}";
return field_name_and_values.str();
}
field_name_and_values << '&' << *field_name_iterator << ':' << field4;
field_name_and_values << "}";
return field_name_and_values.str();
}
// This specialization of the helper method is a shortcut for EventMetric
// instances with no fields.
template<>
inline std::string MakeFieldNameString<Unused, Unused, Unused, Unused>(
const std::vector<std::string>& /* field_names */,
const Unused /* unused1 */, const Unused /* unused2 */,
const Unused /* unused3 */, const Unused /* unused4 */) {
return "";
}
// This helper function appends the field names to a vector of strings.
inline void AppendFieldNames(std::vector<std::string>* field_name_vector,
int field_count, ...) {
va_list field_names;
va_start(field_names, field_count);
for (int x = 0; x < field_count; x++) {
field_name_vector->push_back(va_arg(field_names, const char*));
}
va_end(field_names);
}
// These helper methods and FirstUnusedType assure that there is no mismatch
// between the specified types for EventMetrics and the constructors and
// methods used for the specializations.
template <bool>
struct CompileAssert {};
#define COMPILE_ASSERT(expr, msg) \
typedef impl::CompileAssert<(bool(expr))> msg[bool(expr) ? 1 : -1]
template <typename T> struct is_unused { static const bool value = false; };
template <> struct is_unused<Unused> { static const bool value = true; };
template <typename F1, typename F2, typename F3, typename F4>
class FirstUnusedType {
static const bool a = is_unused<F1>::value;
static const bool b = is_unused<F2>::value;
static const bool c = is_unused<F3>::value;
static const bool d = is_unused<F4>::value;
// Check that all types after the first Unused are also Unused.
COMPILE_ASSERT(a <= b, Invalid_Unused_At_Position_2);
COMPILE_ASSERT(b <= c, Invalid_Unused_At_Position_3);
COMPILE_ASSERT(c <= d, Invalid_Unused_At_Position_4);
public:
static const int value = 5 - (a + b + c + d);
};
// Asserts that no Unused types exist before N; after N, are all Unused types.
#define ASSERT_METRIC_UNUSED_START_FROM(N) \
COMPILE_ASSERT((\
impl::FirstUnusedType<F1, F2, F3, F4>::value) == N, \
Unused_Start_From_##N)
} // namespace impl
// Overloaded template constructor implementations for EventMetric.
template<typename F1, typename F2, typename F3, typename F4>
EventMetric<F1, F2, F3, F4>::EventMetric(
@@ -286,8 +176,8 @@ EventMetric<F1, F2, F3, F4>::EventMetric(
const char* field_name)
: BaseEventMetric(metric_name) {
ASSERT_METRIC_UNUSED_START_FROM(2);
impl::AppendFieldNames(&field_names_,
impl::FirstUnusedType<F1, F2, F3, F4>::value - 1,
util::AppendFieldNames(&field_names_,
util::FirstUnusedType<F1, F2, F3, F4>::value - 1,
field_name);
}
template<typename F1, typename F2, typename F3, typename F4>
@@ -297,8 +187,8 @@ EventMetric<F1, F2, F3, F4>::EventMetric(
const char* field_name2)
: BaseEventMetric(metric_name) {
ASSERT_METRIC_UNUSED_START_FROM(3);
impl::AppendFieldNames(&field_names_,
impl::FirstUnusedType<F1, F2, F3, F4>::value - 1,
util::AppendFieldNames(&field_names_,
util::FirstUnusedType<F1, F2, F3, F4>::value - 1,
field_name1, field_name2);
}
template<typename F1, typename F2, typename F3, typename F4>
@@ -309,8 +199,8 @@ EventMetric<F1, F2, F3, F4>::EventMetric(
const char* field_name3)
: BaseEventMetric(metric_name) {
ASSERT_METRIC_UNUSED_START_FROM(4);
impl::AppendFieldNames(&field_names_,
impl::FirstUnusedType<F1, F2, F3, F4>::value - 1,
util::AppendFieldNames(&field_names_,
util::FirstUnusedType<F1, F2, F3, F4>::value - 1,
field_name1, field_name2, field_name3);
}
template<typename F1, typename F2, typename F3, typename F4>
@@ -322,8 +212,8 @@ EventMetric<F1, F2, F3, F4>::EventMetric(
const char* field_name4)
: BaseEventMetric(metric_name) {
ASSERT_METRIC_UNUSED_START_FROM(5);
impl::AppendFieldNames(&field_names_,
impl::FirstUnusedType<F1, F2, F3, F4>::value - 1,
util::AppendFieldNames(&field_names_,
util::FirstUnusedType<F1, F2, F3, F4>::value - 1,
field_name1, field_name2,
field_name3, field_name4);
}
@@ -332,7 +222,7 @@ template<typename F1, typename F2, typename F3, typename F4>
void EventMetric<F1, F2, F3, F4>::Record(
double value, F1 field1, F2 field2, F3 field3, F4 field4) {
std::string field_name_values =
impl::MakeFieldNameString(field_names_, field1, field2, field3, field4);
util::MakeFieldNameString(field_names_, field1, field2, field3, field4);
BaseEventMetric::Record(field_name_values, value);
}