Add reach only shuffle and aggregation phase. #1600

ple13 · 2024-05-02T02:21:46Z

No description provided.

wfa-reviewable · 2024-05-02T02:21:52Z

This change is

SanjayVas

Reviewed 10 of 10 files at r1, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @kungfucraig, @ple13, and @renjiezh)

MODULE.bazel line 328 at r1 (raw file):

)

archive_override(

nit: Add a DO_NOT_SUBMIT comment referencing the any-sketch PR so it's

Clear that this is intended to be changed before submitting/merging, and
Clear which pending PR this code comes from.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 200 at r1 (raw file):

  RETURN_IF_ERROR(VerifySketchParameters(request.sketch_params()));

  // Verify that the ring modulus is a prime.

nit: Extract an IsPrime bare function. This way that function could be independently changed to e.g. use a more efficient implementation. It also makes the code self-documenting, making the comment unnecessary.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 477 at r1 (raw file):

  // Count the non-zero registers.
  int64_t non_empty_register_count = 0;
  for (auto x : combined_sketch) {

nit: qualify auto as much as possible. In this case, I believe it can be const auto, since nothing will be mutating the copy of the integer.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 477 at r1 (raw file):

  // Count the non-zero registers.
  int64_t non_empty_register_count = 0;
  for (auto x : combined_sketch) {

nit: prefer meaningful variable names. The common exception is loop indices, but this isn't an index.

Code quote:

kungfucraig

Reviewed 10 of 10 files at r1, all commit messages.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @ple13 and @renjiezh)

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.h line 28 at r1 (raw file):

    CompleteShufflePhaseResponse;

absl::StatusOr<CompleteShufflePhaseResponse> CompleteShufflePhase(

nit: Can you add comments that Complete Shuffle Phase and CompleteAggregationPhase apply to Reach and Frequency measurements? You could consider renaming them in a follow-up PR.

ple13

Reviewable status: 1 of 15 files reviewed, 2 unresolved discussions (waiting on @kungfucraig, @renjiezh, and @SanjayVas)

MODULE.bazel line 328 at r1 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

nit: Add a DO_NOT_SUBMIT comment referencing the any-sketch PR so it's

Clear that this is intended to be changed before submitting/merging, and

Clear which pending PR this code comes from.

Thanks. I've cut a release for any-sketch and it's ready now.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.h line 28 at r1 (raw file):

Previously, kungfucraig (Craig Wright) wrote…

nit: Can you add comments that Complete Shuffle Phase and CompleteAggregationPhase apply to Reach and Frequency measurements? You could consider renaming them in a follow-up PR.

I've changed the name of the functions.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 200 at r1 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

nit: Extract an IsPrime bare function. This way that function could be independently changed to e.g. use a more efficient implementation. It also makes the code self-documenting, making the comment unnecessary.

Done.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 477 at r1 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

nit: qualify auto as much as possible. In this case, I believe it can be const auto, since nothing will be mutating the copy of the integer.

Done.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 477 at r1 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

nit: prefer meaningful variable names. The common exception is loop indices, but this isn't an index.

Done.

benjaminkreuter · 2024-05-03T14:46:26Z

...a/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc

@@ -72,9 +72,26 @@ absl::Status VerifySketchParameters(const ShareShuffleSketchParams& params) {
  return absl::OkStatus();
 }

+// Checks if val is a prime.
+absl::StatusOr<bool> IsPrime(int val) {


This is an inefficient way to check primality. It should be OK given how it is used, but we should be careful with this function (especially if we ever need a larger modulus); can you add a note about the O(sqrt(val)) running time to the comment above?

benjaminkreuter · 2024-05-03T14:53:34Z

...a/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc

+    }
+    for (int j = 0; j < sketch_size; j++) {
+      // It's guaranteed that (combined_sketch[j] + share_vector[j]) is
+      // not greater than 2^{32}-1.


I think it is worth having a CHECK here.

benjaminkreuter · 2024-05-03T15:01:49Z

...a/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc

+  // Transform share of non-zero registers to share of a non-zero random value.
+  for (int j = 0; j < sketch_size; j++) {
+    combined_sketch[j] =
+        (combined_sketch[j] * r[j]) % request.sketch_params().ring_modulus();


Maybe it is also worth having a CHECK here? As an alternative we can use 64 bit arithmetic here e.g. change the type of 'r' to 'std::vector<uint64_t>'.

SanjayVas

Reviewed 15 of 15 files at r2, all commit messages.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @kungfucraig, @ple13, and @renjiezh)

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 77 at r2 (raw file):

// Checks if val is a prime.
absl::StatusOr<bool> IsPrime(int val) {
  if (val < 0) {

Just return false in this case so the return type can be bool. It's not incorrect to ask if a negative integer is prime, it's just that the answer is always "no".

ple13

Reviewable status: 14 of 15 files reviewed, 5 unresolved discussions (waiting on @benjaminkreuter, @kungfucraig, @renjiezh, and @SanjayVas)

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 76 at r2 (raw file):

Previously, benjaminkreuter wrote…

This is an inefficient way to check primality. It should be OK given how it is used, but we should be careful with this function (especially if we ever need a larger modulus); can you add a note about the O(sqrt(val)) running time to the comment above?

Done.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 77 at r2 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

Just return false in this case so the return type can be bool. It's not incorrect to ask if a negative integer is prime, it's just that the answer is always "no".

Done.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 246 at r2 (raw file):

Previously, benjaminkreuter wrote…

I think it is worth having a CHECK here.

I've replaced it with a util function that handles the overflow.

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 268 at r2 (raw file):

Previously, benjaminkreuter wrote…

Maybe it is also worth having a CHECK here? As an alternative we can use 64 bit arithmetic here e.g. change the type of 'r' to 'std::vector<uint64_t>'.

Thanks Ben for catching this! I've updated the code.

kungfucraig

Reviewed 14 of 15 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @benjaminkreuter, @renjiezh, and @SanjayVas)

SanjayVas

Reviewed 1 of 1 files at r3, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @benjaminkreuter and @renjiezh)

stevenwarejones

Reviewed 14 of 15 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @benjaminkreuter and @renjiezh)

stevenwarejones

Reviewed 2 of 2 files at r4, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @benjaminkreuter and @renjiezh)

renjiezh

Reviewed 7 of 15 files at r2, 2 of 2 files at r4, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @benjaminkreuter and @ple13)

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 223 at r4 (raw file):

  }

  const int sketch_size = request.sketch_params().register_count();

Do we need to check if sketch_size is 0?

ple13

Reviewable status: 13 of 15 files reviewed, 4 unresolved discussions (waiting on @benjaminkreuter, @kungfucraig, @renjiezh, @SanjayVas, and @stevenwarejones)

src/main/cc/wfa/measurement/internal/duchy/protocol/share_shuffle/honest_majority_share_shuffle_utility.cc line 223 at r4 (raw file):

Previously, renjiezh wrote…

Do we need to check if sketch_size is 0?

Yeah, I've added the check.

Add reach only shuffle and aggregation phase.

e5feede

ple13 requested review from kungfucraig, SanjayVas and renjiezh May 2, 2024 02:27

SanjayVas approved these changes May 2, 2024

View reviewed changes

kungfucraig approved these changes May 2, 2024

View reviewed changes

ple13 added 3 commits May 3, 2024 18:10

Address comments.

7f9a804

Resolve conflicts.

7f3eff6

Fix Lint.

9a86ebc

ple13 commented May 3, 2024

View reviewed changes

benjaminkreuter reviewed May 3, 2024

View reviewed changes

SanjayVas requested a review from kungfucraig May 3, 2024 19:38

SanjayVas requested changes May 3, 2024

View reviewed changes

Address comments.

43a43c4

ple13 commented May 3, 2024

View reviewed changes

kungfucraig approved these changes May 3, 2024

View reviewed changes

SanjayVas approved these changes May 3, 2024

View reviewed changes

ple13 requested a review from stevenwarejones May 3, 2024 21:23

stevenwarejones approved these changes May 6, 2024

View reviewed changes

Merge main branch into this branch.

45095fa

stevenwarejones approved these changes May 10, 2024

View reviewed changes

renjiezh requested changes May 15, 2024

View reviewed changes

Verify sketch params in aggregation phase.

08530ca

ple13 commented May 15, 2024

View reviewed changes

Merge branch 'main' into lephi-implement-cpp-reach-only-hmss

e7c0230

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add reach only shuffle and aggregation phase. #1600

Add reach only shuffle and aggregation phase. #1600

ple13 commented May 2, 2024

wfa-reviewable commented May 2, 2024

SanjayVas left a comment

kungfucraig left a comment

ple13 left a comment

benjaminkreuter May 3, 2024

benjaminkreuter May 3, 2024

benjaminkreuter May 3, 2024

SanjayVas left a comment

ple13 left a comment

kungfucraig left a comment

SanjayVas left a comment

stevenwarejones left a comment

stevenwarejones left a comment

renjiezh left a comment

ple13 left a comment

Add reach only shuffle and aggregation phase. #1600

Are you sure you want to change the base?

Add reach only shuffle and aggregation phase. #1600

Conversation

ple13 commented May 2, 2024

wfa-reviewable commented May 2, 2024

SanjayVas left a comment

Choose a reason for hiding this comment

kungfucraig left a comment

Choose a reason for hiding this comment

ple13 left a comment

Choose a reason for hiding this comment

benjaminkreuter May 3, 2024

Choose a reason for hiding this comment

benjaminkreuter May 3, 2024

Choose a reason for hiding this comment

benjaminkreuter May 3, 2024

Choose a reason for hiding this comment

SanjayVas left a comment

Choose a reason for hiding this comment

ple13 left a comment

Choose a reason for hiding this comment

kungfucraig left a comment

Choose a reason for hiding this comment

SanjayVas left a comment

Choose a reason for hiding this comment

stevenwarejones left a comment

Choose a reason for hiding this comment

stevenwarejones left a comment

Choose a reason for hiding this comment

renjiezh left a comment

Choose a reason for hiding this comment

ple13 left a comment

Choose a reason for hiding this comment