v0.5.5

This is primarily a bug fix release to address #1616

What's Changed

• Add corresponding seed encryption key in HMSS requisitions. by @renjiezh in #1585
• Fix swapped names for AWS Duchy IP addresses. by @SanjayVas in #1605
• Implement FrequencyVectorBuilder by @kungfucraig in #1604
• Add default service config by @jcorilla in #1603
• Add requisition fulfillment for hmss. by @ple13 in #1567
• Move SecretSeed from database to RequisitionDetails proto. by @renjiezh in #1592
• Support head environment in workflows. by @SanjayVas in #1608
• Enable nightly deployments to head environment. by @SanjayVas in #1609
• Fix setting of Metric state to SUCCEEDED by @tristanvuong2021 in #1593
• Move shareshuffle files to v2alpha subdirectory by @kungfucraig in #1614
• Factor InMemoryVidIndexMap out of VidIndexMap by @kungfucraig in #1615

Full Changelog: v0.5.4...v0.5.5

v0.5.4

This is primarily a bug fix release to address #1596

Potentially Requiring Action

• The dev configuration for EKS now manages external IP addresses with Terraform. See #1586
  • Use terraform import to import your existing Elastic IP address resources to avoid unintentional external IP address changes.

What's Changed

• Skip pushing images in Update CMMS workflow if there's an existing image tag. by @SanjayVas in #1582
• Manage external IP addresses for EKS in Terraform. by @SanjayVas in #1586
• Add back dispatch for image push. by @renjiezh in #1598
• Add INITIALIZED stage for aggregator in HMSS. by @renjiezh in #1587
• Fail correctness/integration test if Measurement is cancelled. by @SanjayVas in #1599
• Populate MeasurementSpec.serialized_measurement_public_key field. by @SanjayVas in #1597

Full Changelog: v0.5.3...v0.5.4

v0.5.3

This is primarily a bug fix release to address #1588

Known Issues

• Requisitions created by the Measurement CLI or Reporting system are not backwards compatible with v0.4.x DataProviders. See #1596

Potentially Requiring Action

• The dev configuration for GKE now manages external IP addresses with Terraform. See #1583
  • Clusters using the dev configuration must be running GKE 1.29+
  • Service resources may need to be manually deleted as migration of existing Services to L4-RBS is not supported.
  • Use terraform import and specify existing IP addresses in your Terraform config to avoid unintentional external IP address changes. See #1590

What's Changed

• Manage external IP addresses for GKE in Terraform. by @SanjayVas in #1583
• Add variables for specifying existing IP addresses in gcloud Terraform modules. by @SanjayVas in #1590
• Populate Requisition.encrypted_requisition_spec_ciphertext field. by @SanjayVas in #1589

Full Changelog: v0.5.2...v0.5.3

v0.5.3-rc1

This is primarily a bug fix release to address #1588

Potentially Requiring Action

• The dev configuration for GKE now manages external IP addresses with Terraform. See #1583
  • Clusters using the dev configuration must be running GKE 1.29+
  • Use terraform import and specify existing IP addresses in your Terraform config to avoid unintentional external IP address changes. See #1590

What's Changed

• Manage external IP addresses for GKE in Terraform. by @SanjayVas in #1583
• Add variables for specifying existing IP addresses in gcloud Terraform modules. by @SanjayVas in #1590
• Populate Requisition.encrypted_requisition_spec_ciphertext field. by @SanjayVas in #1589

Full Changelog: v0.5.2...v0.5.3-rc1

v0.5.2

Starting with these release, scanning alerts for container image vulnerabilities should appear in GitHub Security.

Highlights

• Closed #1544
• Closed #1553
• Various performance improvements for Reporting API operations.
• Updated Java base images.
• More detailed errors from many Kingdom public API services (ongoing).

Potentially Requiring Action

• CMMS instances may define a set of known EventGroup metadata types that are shared between components and integrators. These are the only EventGroup metadata types that can be used by multiple DataProviders. See #1512
  • The Kingdom internal API server and the Reporting v1 and v2 public API servers have a new --known-event-group-metadata-type option for specifying a FileDescriptorSet for these known metadata types.
    • The dev configuration expects this to come from a known_event_group_metadata_type_set.pb file in the config-files ConfigMap. By default, this contains the types defined in simulator_synthetic_data_spec.proto and test_metadata_message.proto
• There is a new required --requisition-fulfillment-service-duchy-id option to the EDP simulator. This is in preparation for supporting the HMSS protocol. See #1574.
  • This option, along with the --requisition-fulfillment-service-target and --requisition-fulfillment-service-cert-host option, form an argument group that can be repeated to allow fulfillment at multiple Duchies.

What's Changed

• Update cross-media-measurement-api dep for CustomMaximumFrequencyPerUser in DeterministicCount methodology by @iverson52000 in #1462
• Increase Mill and EDP simulator memory sizes for dev configuration. by @SanjayVas in #1514
• Remove unnecessary hard-coded delay from K8s panel match test. by @SanjayVas in #1515
• Move reporting sets call out of build internal create metric request method by @tristanvuong2021 in #1499
• Fix race conditions in empty cluster K8s tests. by @SanjayVas in #1517
• Duchy Herald supports HMSS protocol by @renjiezh in #1513
• Introduce the concept of known EventGroup metadata types. by @SanjayVas in #1512
• Implement the aggregation phase of the HMSS protocol. by @ple13 in #1494
• Replace one insert call per row with one insert call for multiple rows by @tristanvuong2021 in #1502
• Replace one update call per row with one update call for multiple rows by @tristanvuong2021 in #1518
• Fix order of batch get measurements response by @tristanvuong2021 in #1519
• Change all Internal Reporting Measurements Service Methods responses to empty responses by @tristanvuong2021 in #1521
• Add internal BatchGetDataProviders. by @SanjayVas in #1523
• Update cross-media-measurement-api dep for DataProvider capabilities. by @SanjayVas in #1472
• Update simulator guides to reflect BigQuery event source changes. by @SanjayVas in #1522
• Update common-jvm dep for base Java image update. by @SanjayVas in #1525
• Add data-providers update-capabilities subcommand to MeasurementSystem CLI. by @SanjayVas in #1528
• Use remote cache when building protobufs for API linter. by @SanjayVas in #1531
• Use values when logging external IDs by @jcorilla in #1532
• Updates Terraform and K8s to support deploy to AWS QA env by @YuhongWang-Amazon in #1534
• Fix bazel container script by @YuhongWang-Amazon in #1536
• Add detailed conversion of internal to external errors by @jcorilla in #1520
• Add detailed logging for public api keys service by @jcorilla in #1541
• Add detailed logging to public accounts service by @jcorilla in #1540
• Add detailed logging for public certificate service by @jcorilla in #1542
• Update local deployment instruction. by @renjiezh in #1543
• Update any-sketch to 0.8 by @kungfucraig in #1554
• Fix dev Duchy kustomization_dir targets not being marked as testonly. by @SanjayVas in #1547
• Update common-jvm dep for grpc-java and netty-codec-http2. by @SanjayVas in #1555
• Add detailed logging to population service by @jcorilla in #1548
• Add detailed logging for public keys service by @jcorilla in #1549
• Update workflows for QA environment. by @SanjayVas in #1546
• Add Open Telemetry Auto Instrumented Traces to Cloud Trace by @tristanvuong2021 in #1537
• Implement HonestMajorityShareShuffle mill by @renjiezh in #1506
• Add stubs for a set of utilities for building a FrequencyVector. by @kungfucraig in #1552
• Add the function to generate hmss sketch. by @ple13 in #1509
• Add State to Internal Metric by @tristanvuong2021 in #1557
• Update common-jvm dep for changes to java_image. by @SanjayVas in #1561
• Improve performance of batch create measurements by @tristanvuong2021 in #1533
• Add CreateUiReport to the CLI by @bdomen-ggl in #1535
• Add detailed logging to requisitions service by @jcorilla in #1550
• Add detailed logging for exchanges service by @jcorilla in #1556
• Add index for update metric calculation spec reporting metrics in create metrics by @tristanvuong2021 in #1538
• Add getUserVirtualIdUniverse() to EventQuery by @renjiezh in #1562
• Add detailed logging to ExchangeSteps and Attempts services by @jcorilla in #1565
• Scan container images with Trivy on release. by @SanjayVas in #1558
• Update Building doc with more recent information. by @SanjayVas in #1572
• Refactor Metric Reuse to handle Metric State by @tristanvuong2021 in #1564
• Add population attribute annotations by @kungfucraig in #1571
• Add detailed logging to data providers service by @jcorilla in #1568
• Add detailed logging to measurement c...

v0.5.2-rc3

Starting with these release, scanning alerts for container image vulnerabilities should appear in GitHub Security.

Highlights

• Closed #1544
• Closed #1553
• Various performance improvements for Reporting API operations.
• Updated Java base images.
• More detailed errors from many Kingdom public API services (ongoing).

Potentially Requiring Action

• CMMS instances may define a set of known EventGroup metadata types that are shared between components and integrators. These are the only EventGroup metadata types that can be used by multiple DataProviders. See #1512
  • The Kingdom internal API server and the Reporting v1 and v2 public API servers have a new --known-event-group-metadata-type option for specifying a FileDescriptorSet for these known metadata types.
    • The dev configuration expects this to come from a known_event_group_metadata_type_set.pb file in the config-files ConfigMap. By default, this contains the types defined in simulator_synthetic_data_spec.proto and test_metadata_message.proto
• There is a new required --requisition-fulfillment-service-duchy-id option to the EDP simulator. This is in preparation for supporting the HMSS protocol. See #1574.
  • This option, along with the --requisition-fulfillment-service-target and --requisition-fulfillment-service-cert-host option, form an argument group that can be repeated to allow fulfillment at multiple Duchies.

What's Changed

• Update cross-media-measurement-api dep for CustomMaximumFrequencyPerUser in DeterministicCount methodology by @iverson52000 in #1462
• Increase Mill and EDP simulator memory sizes for dev configuration. by @SanjayVas in #1514
• Remove unnecessary hard-coded delay from K8s panel match test. by @SanjayVas in #1515
• Move reporting sets call out of build internal create metric request method by @tristanvuong2021 in #1499
• Fix race conditions in empty cluster K8s tests. by @SanjayVas in #1517
• Duchy Herald supports HMSS protocol by @renjiezh in #1513
• Introduce the concept of known EventGroup metadata types. by @SanjayVas in #1512
• Implement the aggregation phase of the HMSS protocol. by @ple13 in #1494
• Replace one insert call per row with one insert call for multiple rows by @tristanvuong2021 in #1502
• Replace one update call per row with one update call for multiple rows by @tristanvuong2021 in #1518
• Fix order of batch get measurements response by @tristanvuong2021 in #1519
• Change all Internal Reporting Measurements Service Methods responses to empty responses by @tristanvuong2021 in #1521
• Add internal BatchGetDataProviders. by @SanjayVas in #1523
• Update cross-media-measurement-api dep for DataProvider capabilities. by @SanjayVas in #1472
• Update simulator guides to reflect BigQuery event source changes. by @SanjayVas in #1522
• Update common-jvm dep for base Java image update. by @SanjayVas in #1525
• Add data-providers update-capabilities subcommand to MeasurementSystem CLI. by @SanjayVas in #1528
• Use remote cache when building protobufs for API linter. by @SanjayVas in #1531
• Use values when logging external IDs by @jcorilla in #1532
• Updates Terraform and K8s to support deploy to AWS QA env by @YuhongWang-Amazon in #1534
• Fix bazel container script by @YuhongWang-Amazon in #1536
• Add detailed conversion of internal to external errors by @jcorilla in #1520
• Add detailed logging for public api keys service by @jcorilla in #1541
• Add detailed logging to public accounts service by @jcorilla in #1540
• Add detailed logging for public certificate service by @jcorilla in #1542
• Update local deployment instruction. by @renjiezh in #1543
• Update any-sketch to 0.8 by @kungfucraig in #1554
• Fix dev Duchy kustomization_dir targets not being marked as testonly. by @SanjayVas in #1547
• Update common-jvm dep for grpc-java and netty-codec-http2. by @SanjayVas in #1555
• Add detailed logging to population service by @jcorilla in #1548
• Add detailed logging for public keys service by @jcorilla in #1549
• Update workflows for QA environment. by @SanjayVas in #1546
• Add Open Telemetry Auto Instrumented Traces to Cloud Trace by @tristanvuong2021 in #1537
• Implement HonestMajorityShareShuffle mill by @renjiezh in #1506
• Add stubs for a set of utilities for building a FrequencyVector. by @kungfucraig in #1552
• Add the function to generate hmss sketch. by @ple13 in #1509
• Add State to Internal Metric by @tristanvuong2021 in #1557
• Update common-jvm dep for changes to java_image. by @SanjayVas in #1561
• Improve performance of batch create measurements by @tristanvuong2021 in #1533
• Add CreateUiReport to the CLI by @bdomen-ggl in #1535
• Add detailed logging to requisitions service by @jcorilla in #1550
• Add detailed logging for exchanges service by @jcorilla in #1556
• Add index for update metric calculation spec reporting metrics in create metrics by @tristanvuong2021 in #1538
• Add getUserVirtualIdUniverse() to EventQuery by @renjiezh in #1562
• Add detailed logging to ExchangeSteps and Attempts services by @jcorilla in #1565
• Scan container images with Trivy on release. by @SanjayVas in #1558
• Update Building doc with more recent information. by @SanjayVas in #1572
• Refactor Metric Reuse to handle Metric State by @tristanvuong2021 in #1564
• Add population attribute annotations by @kungfucraig in #1571
• Add detailed logging to data providers service by @jcorilla in #1568
• Add detailed logging to measureme...

v0.5.2-rc2

Starting with these release, scanning alerts for container image vulnerabilities should appear in GitHub Security.

Highlights

• Closed #1544
• Closed #1553
• Various performance improvements for Reporting API operations.
• Updated Java base images.
• More detailed errors from many Kingdom public API services (ongoing).

Potentially Requiring Action

• CMMS instances may define a set of known EventGroup metadata types that are shared between components and integrators. These are the only EventGroup metadata types that can be used by multiple DataProviders. See #1512
  • The Kingdom internal API server and the Reporting v1 and v2 public API servers have a new --known-event-group-metadata-type option for specifying a FileDescriptorSet for these known metadata types.
    • The dev configuration expects this to come from a known_event_group_metadata_type_set.pb file in the config-files ConfigMap. By default, this contains the types defined in simulator_synthetic_data_spec.proto and test_metadata_message.proto

What's Changed

• Update cross-media-measurement-api dep for CustomMaximumFrequencyPerUser in DeterministicCount methodology by @iverson52000 in #1462
• Increase Mill and EDP simulator memory sizes for dev configuration. by @SanjayVas in #1514
• Remove unnecessary hard-coded delay from K8s panel match test. by @SanjayVas in #1515
• Move reporting sets call out of build internal create metric request method by @tristanvuong2021 in #1499
• Fix race conditions in empty cluster K8s tests. by @SanjayVas in #1517
• Duchy Herald supports HMSS protocol by @renjiezh in #1513
• Introduce the concept of known EventGroup metadata types. by @SanjayVas in #1512
• Implement the aggregation phase of the HMSS protocol. by @ple13 in #1494
• Replace one insert call per row with one insert call for multiple rows by @tristanvuong2021 in #1502
• Replace one update call per row with one update call for multiple rows by @tristanvuong2021 in #1518
• Fix order of batch get measurements response by @tristanvuong2021 in #1519
• Change all Internal Reporting Measurements Service Methods responses to empty responses by @tristanvuong2021 in #1521
• Add internal BatchGetDataProviders. by @SanjayVas in #1523
• Update cross-media-measurement-api dep for DataProvider capabilities. by @SanjayVas in #1472
• Update simulator guides to reflect BigQuery event source changes. by @SanjayVas in #1522
• Update common-jvm dep for base Java image update. by @SanjayVas in #1525
• Add data-providers update-capabilities subcommand to MeasurementSystem CLI. by @SanjayVas in #1528
• Use remote cache when building protobufs for API linter. by @SanjayVas in #1531
• Use values when logging external IDs by @jcorilla in #1532
• Updates Terraform and K8s to support deploy to AWS QA env by @YuhongWang-Amazon in #1534
• Fix bazel container script by @YuhongWang-Amazon in #1536
• Add detailed conversion of internal to external errors by @jcorilla in #1520
• Add detailed logging for public api keys service by @jcorilla in #1541
• Add detailed logging to public accounts service by @jcorilla in #1540
• Add detailed logging for public certificate service by @jcorilla in #1542
• Update local deployment instruction. by @renjiezh in #1543
• Update any-sketch to 0.8 by @kungfucraig in #1554
• Fix dev Duchy kustomization_dir targets not being marked as testonly. by @SanjayVas in #1547
• Update common-jvm dep for grpc-java and netty-codec-http2. by @SanjayVas in #1555
• Add detailed logging to population service by @jcorilla in #1548
• Add detailed logging for public keys service by @jcorilla in #1549
• Update workflows for QA environment. by @SanjayVas in #1546
• Add Open Telemetry Auto Instrumented Traces to Cloud Trace by @tristanvuong2021 in #1537
• Implement HonestMajorityShareShuffle mill by @renjiezh in #1506
• Add stubs for a set of utilities for building a FrequencyVector. by @kungfucraig in #1552
• Add the function to generate hmss sketch. by @ple13 in #1509
• Add State to Internal Metric by @tristanvuong2021 in #1557
• Update common-jvm dep for changes to java_image. by @SanjayVas in #1561
• Improve performance of batch create measurements by @tristanvuong2021 in #1533
• Add CreateUiReport to the CLI by @bdomen-ggl in #1535
• Add detailed logging to requisitions service by @jcorilla in #1550
• Add detailed logging for exchanges service by @jcorilla in #1556
• Add index for update metric calculation spec reporting metrics in create metrics by @tristanvuong2021 in #1538
• Add getUserVirtualIdUniverse() to EventQuery by @renjiezh in #1562
• Add detailed logging to ExchangeSteps and Attempts services by @jcorilla in #1565
• Scan container images with Trivy on release. by @SanjayVas in #1558
• Update Building doc with more recent information. by @SanjayVas in #1572
• Refactor Metric Reuse to handle Metric State by @tristanvuong2021 in #1564
• Add population attribute annotations by @kungfucraig in #1571
• Add detailed logging to data providers service by @jcorilla in #1568
• Add detailed logging to measurement consumers service by @jcorilla in #1569
• Update common-jvm dep for base images. by @SanjayVas in #1575
• Add detailed logging to event groups and metadata descriptors services by @jcorilla in https://github.com/world-federation...

v0.5.2-rc1

Starting with these release, scanning alerts for container image vulnerabilities should appear in GitHub Security.

Highlights

• Closed #1544
• Closed #1553
• Various performance improvements for Reporting API operations.
• Updated Java base images.
• More detailed errors from many Kingdom public API services (ongoing).

Potentially Requiring Action

• CMMS instances may define a set of known EventGroup metadata types that are shared between components and integrators. These are the only EventGroup metadata types that can be used by multiple DataProviders. See #1512
  • The Kingdom internal API server and the Reporting v1 and v2 public API servers have a new --known-event-group-metadata-type option for specifying a FileDescriptorSet for these known metadata types.
    • The dev configuration expects this to come from a known_event_group_metadata_type_set.pb file in the config-files ConfigMap. By default, this contains the types defined in simulator_synthetic_data_spec.proto and test_metadata_message.proto

What's Changed

• Update cross-media-measurement-api dep for CustomMaximumFrequencyPerUser in DeterministicCount methodology by @iverson52000 in #1462
• Increase Mill and EDP simulator memory sizes for dev configuration. by @SanjayVas in #1514
• Remove unnecessary hard-coded delay from K8s panel match test. by @SanjayVas in #1515
• Move reporting sets call out of build internal create metric request method by @tristanvuong2021 in #1499
• Fix race conditions in empty cluster K8s tests. by @SanjayVas in #1517
• Duchy Herald supports HMSS protocol by @renjiezh in #1513
• Introduce the concept of known EventGroup metadata types. by @SanjayVas in #1512
• Implement the aggregation phase of the HMSS protocol. by @ple13 in #1494
• Replace one insert call per row with one insert call for multiple rows by @tristanvuong2021 in #1502
• Replace one update call per row with one update call for multiple rows by @tristanvuong2021 in #1518
• Fix order of batch get measurements response by @tristanvuong2021 in #1519
• Change all Internal Reporting Measurements Service Methods responses to empty responses by @tristanvuong2021 in #1521
• Add internal BatchGetDataProviders. by @SanjayVas in #1523
• Update cross-media-measurement-api dep for DataProvider capabilities. by @SanjayVas in #1472
• Update simulator guides to reflect BigQuery event source changes. by @SanjayVas in #1522
• Update common-jvm dep for base Java image update. by @SanjayVas in #1525
• Add data-providers update-capabilities subcommand to MeasurementSystem CLI. by @SanjayVas in #1528
• Use remote cache when building protobufs for API linter. by @SanjayVas in #1531
• Use values when logging external IDs by @jcorilla in #1532
• Updates Terraform and K8s to support deploy to AWS QA env by @YuhongWang-Amazon in #1534
• Fix bazel container script by @YuhongWang-Amazon in #1536
• Add detailed conversion of internal to external errors by @jcorilla in #1520
• Add detailed logging for public api keys service by @jcorilla in #1541
• Add detailed logging to public accounts service by @jcorilla in #1540
• Add detailed logging for public certificate service by @jcorilla in #1542
• Update local deployment instruction. by @renjiezh in #1543
• Update any-sketch to 0.8 by @kungfucraig in #1554
• Fix dev Duchy kustomization_dir targets not being marked as testonly. by @SanjayVas in #1547
• Update common-jvm dep for grpc-java and netty-codec-http2. by @SanjayVas in #1555
• Add detailed logging to population service by @jcorilla in #1548
• Add detailed logging for public keys service by @jcorilla in #1549
• Update workflows for QA environment. by @SanjayVas in #1546
• Add Open Telemetry Auto Instrumented Traces to Cloud Trace by @tristanvuong2021 in #1537
• Implement HonestMajorityShareShuffle mill by @renjiezh in #1506
• Add stubs for a set of utilities for building a FrequencyVector. by @kungfucraig in #1552
• Add the function to generate hmss sketch. by @ple13 in #1509
• Add State to Internal Metric by @tristanvuong2021 in #1557
• Update common-jvm dep for changes to java_image. by @SanjayVas in #1561
• Improve performance of batch create measurements by @tristanvuong2021 in #1533
• Add CreateUiReport to the CLI by @bdomen-ggl in #1535
• Add detailed logging to requisitions service by @jcorilla in #1550
• Add detailed logging for exchanges service by @jcorilla in #1556
• Add index for update metric calculation spec reporting metrics in create metrics by @tristanvuong2021 in #1538
• Add getUserVirtualIdUniverse() to EventQuery by @renjiezh in #1562
• Add detailed logging to ExchangeSteps and Attempts services by @jcorilla in #1565
• Scan container images with Trivy on release. by @SanjayVas in #1558
• Update Building doc with more recent information. by @SanjayVas in #1572
• Refactor Metric Reuse to handle Metric State by @tristanvuong2021 in #1564
• Add population attribute annotations by @kungfucraig in #1571
• Add detailed logging to data providers service by @jcorilla in #1568
• Add detailed logging to measurement consumers service by @jcorilla in #1569
• Update common-jvm dep for base images. by @SanjayVas in #1575
• Add detailed logging to event groups and metadata descriptors services by @jcorilla in https://github.com/world-federation...

v0.5.1

This is a bug fix release that primarily affects the configuration for running the Measurement correctness test.

Bugs Fixed

• Dev release workflow does not complete.
• Variance is NaN when reach is small (#1498)

Performance Optimizations

• Only generate synthetic events within specified time range (#1488)
• Refactor SubmitBatchRequests to use Coroutines (#1467)

What's Changed

• Add UI features from Demo work by @bdomen-ggl in #1495
• Fix simulator service account not being created for GKE. by @SanjayVas in #1501
• Output uniformly random guess at frequency distribution when reach is too small by @riemanli in #1498
• Specify auto_minor_version_upgrade = false for AWS Postgres DB. by @SanjayVas in #1504
• Update HMSS stages. Revert Duchy ControlService into Blob-Only Pattern. by @renjiezh in #1476
• Increase heap size for dev requisition fulfillment server. by @SanjayVas in #1503
• Only generate synthetic events within specified time range. by @SanjayVas in #1488
• Increase work lock duration for dev LLv2 mills. by @SanjayVas in #1507
• Increase test timeout for K8s test workflow. by @SanjayVas in #1505
• Refactor SubmitBatchRequests to use Coroutines by @tristanvuong2021 in #1467
• Drop worker2 Duchy and BigQuery vars from gcloud testing Terraform. by @SanjayVas in #1493
• Fix configure-simulators workflow not setting google_cloud_project. by @SanjayVas in #1508

Full Changelog: v0.5.0...v0.5.1

v0.5.0

Known Issues

• The dev K8s configuration for the synthetic generator EDP simulator on GKE has an error. This is fixed in #1501

Highlights

• Reporting v2 is no longer considered a preview. It includes some new features.
  • Report scheduling. See #1234
  • MetricCalculationSpec as a reusable resource type. See #1278
  • Reuse of existing Metric resources during Report creation. See #1347
  • Improved performance using batch methods when calling the CMMS API.
  • Tagging support for some API resources.
  • Metric variance information (to support features such as displaying error bars).
• The Duchy component can now be run on AWS EKS. See the deployment guide.
• There is a new guide covering leaf certificate and encryption key rotation.
• gRPC log entries now include a trace ID when metrics instrumentation is enabled. See #766
  • This can be used to trace calls across servers.
• Structured logging for Java is now supported on GKE. See #762
  • Google Cloud Logging will now show one entry per Java log entry (as opposed to one entry per line) and have the appropriate severity.
• Arbitrary event message types are now supported by the EDP simulator. See #1327
• The EDP simulator supports creation of multiple EventGroups. See #1424
• The MeasurementSystem CLI supports creation of Measurements with type REACH. See #1254

Bugs Fixed

• Certificates using a different signature algorithm than their issuer are not supported. See world-federation-of-advertisers/cross-media-measurement-api#176
• EDP simulator generates NaN values in direct frequency. See #1384

Potentially Requiring Action

• Incompatible database schema and API changes have been made to Reporting v2.
  • Users of the previous preview release will need to wipe their databases before upgrading.
• K8s Workload Identity service account creation for GKE has been moved from Terraform to K8s object configuration. See #1324
• The --event-group-spec option of SyntheticGeneratorEdpSimulatorRunner is now a key-value pair of EventGroup reference ID suffix to SyntheticEventGroupSpec file path. This can be specified multiple times. See #1424
• The BigQuery-backed EDP simulator now uses UK pilot event templates. See #1491
• CLI tool options have changed.

CMMS Public API Changes

The cross-media-measurement-api dependency has been updated to 0.56.0. See world-federation-of-advertisers/cross-media-measurement-api@v0.43.0...v0.56.0 for the full list of changes.

• Signatures and certificates for public key fields have been removed where not needed by the consent signaling protocol. See world-federation-of-advertisers/cross-media-measurement-api#187
  • measurement_consumer_certificate in EventGroup has been removed.
  • measurement_consumer_public_key in EventGroup no longer contains a signature.
  • data_provider_public_key in Measurement.DataProviderEntry.Value no longer contains a signature.
  • data_provider_public_key in Requisition no longer contains a signature.
• Serialized messages now include type URLs. See world-federation-of-advertisers/cross-media-measurement-api#185
  • measurement_consumer_public_key in EventGroup now has type google.protobuf.Any.
    • The old version of this field has been marked as deprecated and is now called signed_measurement_consumer_public_key.
  • data_provider_public_key in Measurement.DataProviderEntry.Value now has type google.protobuf.Any.
  • data_provider_public_key in Requisition now has type google.protobuf.Any.
    • The old version of this field has been marked as deprecated and is now called signed_data_provider_public_key.
  • exchange_workflow in RecurringExchange now has type google.protobuf.Any.
  • serialized_exchange_workflow in ExchangeStep has been replaced with exchange_workflow having type google.protobuf.Any.
  • serialized_measuirement_public_key in MeasurementSpec is deprecated in favor of measurement_public_key having type google.protobuf.Any
    • This field is required for new requests.
  • data_provider_public_key in Requisition now has type google.protobuf.Any.
    • The old version of this field has been marked as deprecated and is now called signed_data_provider_public_key.
  • measurement_public_key in RequisitionSpec now has type google.protobuf.Any.
    • The old version of this field has been marked as deprecated and is now called serialized_measurement_public_key.
  • The SignedData type in the CMMS public API has been renamed to SignedMessage. This is wire-compatible change.
    • This has a new signature_algorithm_oid field.
      • Signers should populate this field.
      • Verifiers must respect this field if populated.
  • Fields of type bytes holding encrypted data now use the EncryptedMessage type instead.
    • encrypted_requisition_spec in Requisition now has type EncryptedMessage.
      • The old version of this field has been marked as deprecated and is now called encrypted_requisition_spec_ciphertext.
    • encrypted_data in FulfillDirectRequisitionRequest has been replaced by encrypted_result having type EncryptedMessage.
      • The old version of this field has been marked as deprecated and is now called encrypted_result_ciphertext.
    • encrypted_metadata in EventGroup now has type EncryptedMessage.
      • The old version of this field has been marked as deprecated and is now called serialized_encrypted_metadata.
• FulfillDirectRequisitionRequest has a new certificate field.
  • Callers should begin setting this field, as it will be required in the future.
• FulfillRequisitionRequest has a new protocol_config field.
  • Callers should begin setting this field, as it will be required in the future.
• DataProvider and EventGroup have a new data_availability_interval field to indicate the range for which data is guaranteed to be available. See world-federation-of-advertisers/cross-media-measurement-api#175
  • There is a new ReplaceDataAvailabilityInterval method for updating this on DataProvider resources.
  • DataProviders should start specifying and updating these fields. They are currently only used by the Report v2 scheduling feature, but will be required in the future.
• Unsupported fields have been removed from ListExchangeStepsRequest.Filter. See world-federation-of-advertisers/cross-media-measurement-api#180
• ClaimReadyExchangeStepRequest has a new required parent field. See world-federation-of-advertisers/cross-media-measurement-api#142
• The data_provider and model_provider fields have been removed from ClaimReadyExchangeStepRequest. See world-federation-of-advertisers/cross-media-measurement-api#142
• The deprecated event_groups field in RequisitionSpec is no longer set by Halo software. See #1277
• maximum_frequency in MeasurementSpec.ReachAndFrequency must be at least 1. See #1276

What's Changed

• s3fromBeam should be false by default by @stevenwarejones in #1240
• Add Reporting V2 Report Scheduling Protos by @tristanvuong2021 in #1234
• Add terraform for AWS worker2 duchy infrastructures by @YuhongWang-Amazon in #1237
• Store measurement computation details in reporting for error bars by @riemanli in #1184
• Add k8s manifests for AWS worker2 duchy by @YuhongWang-Amazon in #1239
• Restrict metric set operations based on metric types by @riemanli in #1247
• Format all Kotlin files with ktfmt 0.46. by @SanjayVas in #1241
• Add github workflows for deploying aws duchy by @YuhongWang-Amazon in #1255
• Remove duplicate filter during event group entry creations by @riemanli in #1245
• Adds an event filter to MetricCalculationSpec. by @kungfucraig in #1250
• Adds key-value pair tags to Report and ReportingSet. by @kungfucraig in #1251
• Update common-jvm dep for Kotlin 1.8. by @SanjayVas in #1235
• Command Line Tool Supports Creation of Reach Measurements. by @renjiezh in #1254
• Squashed commit of the following: by @emadlivani in #1227
• Refactor MeasurementStatistics by @riemanli in https://github.com/world-federation-of-adverti...