orca: create ORCA producer for LB policies to use to receive OOB load reports #5669

dfawley · 2022-09-23T18:13:28Z

RELEASE NOTES:

balancer: add experimental Producer types and methods and * orca: provide a way for LB policies to receive OOB load reports

easwars · 2022-10-05T17:46:06Z

orca/internal/internal.go

@@ -20,8 +20,15 @@
 // avoid polluting the godoc of the top-level orca package.
 package internal

+import ibackoff "google.golang.org/grpc/internal/backoff"


Why do we need this import to be renamed?

We don't; I just like to keep things consistent, and we have a backoff package and an internal/backoff package, and it's confusing if we reference both using the same name in different files.

Package names should ideally be self contained and self explanatory. If we are always going to rename this import, we should ideally rename the package itself.

balancer/balancer.go

easwars · 2022-10-05T18:00:05Z

balancer/balancer.go

+// A Producer is a type shared among potentially many consumers.  It is
+// associated with a SubConn, and an implementation will typically contain
+// other methods to provide additional functionality, e.g. configuration or
+// subscription registration.


This description feels very vague to me, in the sense that I can't get a good idea of when to use a Producer, how to use one etc.

Could you suggest something better after our discussion today?

Ack. The first sentence is fine, but I don't get much technical understanding from: "and an implementation will typically contain other methods to provide additional functionality, e.g. configuration or subscription registration." Additional functionality with respect to what? There's no functionality defined before this. Do we expect this to ever be implemented by another concrete type? This comment is very specific to type producer in orca/producer. If so, why have this interface?

Additional functionality with respect to what? There's no functionality defined before this.

In addition to the builder that created it, I suppose, and the close function the builder returns.

Do we expect this to ever be implemented by another concrete type?

Yes, definitely. This is intended to be generic and able to be implemented by anything. The other example we already have is the health checks, which are implemented in the other languages as a generic wrapper, but hard-coded into gRPC in Go. I'd like to move to a similar model there by using this mechanism.

This comment is very specific to type producer in orca/producer.

Not at all. If it was, it would talk about ORCA configuration. ORCA should not be part of grpc-go directly, but instead should only be a plugin which uses this.

balancer/balancer.go

orca/producer.go

balancer_conn_wrappers.go

orca/producer.go

balancer_conn_wrappers.go

easwars

I think you might have missed pushing a commit. I have more comments which I will add soon.

balancer/balancer.go

zasweq

Cool PR bossman. Some comments.

zasweq · 2022-10-13T18:27:40Z

balancer/balancer.go

+// A Producer is a type shared among potentially many consumers.  It is
+// associated with a SubConn, and an implementation will typically contain
+// other methods to provide additional functionality, e.g. configuration or
+// subscription registration.


Ack. The first sentence is fine, but I don't get much technical understanding from: "and an implementation will typically contain other methods to provide additional functionality, e.g. configuration or subscription registration." Additional functionality with respect to what? There's no functionality defined before this. Do we expect this to ever be implemented by another concrete type? This comment is very specific to type producer in orca/producer. If so, why have this interface?

balancer/balancer.go

balancer_conn_wrappers.go

orca/producer.go

balancer_conn_wrappers.go

orca/producer.go

dfawley

Updated; sorry for the force push; I messed up the synchronization between my laptop & workstation.

dfawley · 2022-10-17T21:06:42Z

balancer/balancer.go

+// A Producer is a type shared among potentially many consumers.  It is
+// associated with a SubConn, and an implementation will typically contain
+// other methods to provide additional functionality, e.g. configuration or
+// subscription registration.


Additional functionality with respect to what? There's no functionality defined before this.

In addition to the builder that created it, I suppose, and the close function the builder returns.

Do we expect this to ever be implemented by another concrete type?

Yes, definitely. This is intended to be generic and able to be implemented by anything. The other example we already have is the health checks, which are implemented in the other languages as a generic wrapper, but hard-coded into gRPC in Go. I'd like to move to a similar model there by using this mechanism.

This comment is very specific to type producer in orca/producer.

Not at all. If it was, it would talk about ORCA configuration. ORCA should not be part of grpc-go directly, but instead should only be a plugin which uses this.

balancer_conn_wrappers.go

internal/testutils/balancer.go

orca/producer.go

zasweq · 2022-10-18T03:10:34Z

orca/producer.go

@@ -197,7 +200,7 @@ func (p *producer) runStream(ctx context.Context) (resetBackoff bool, err error)
 		ReportInterval: durationpb.New(interval),
 	})
 	if err != nil {
-		return resetBackoff, err
+		return false, err


You liked this rather than using default value of bools more?

This was changed due to an earlier comment from @easwars. I said in my reply there I didn't feel strongly either way. I still don't.

zasweq · 2022-10-18T03:21:17Z

orca/producer.go

+	// to avoid overloading a server experiencing problems.  The attempt count
+	// is incremented when stream errors occur and is reset when the stream
+	// reports a result.


I don't think the last part of this comment is correct/like this comment. First of all, reset has two semantical meanings for this field, one for the timer, one for the attempt count, which gets plumbed into Exponential Backoff to determine what the backoff timer gets reset to I'm assuming your reset refers to attempt count. Also, there is no linkage or even local variable named request count. I see backoffAttempt, which isn't a field here, but the int in this function variable. The attempt count is not reset when the stream reports a result. Right now, the attempt count is reset on either a. the minInterval has changed after processing the receive from the stream compared to what was the stream was configured with. b. If the second+ stream.Recv() returns an error, this also feels wrong, I will leave a comment there.

zasweq · 2022-10-18T03:25:42Z

orca/producer.go

+			return resetBackoff, err
+		}
+		resetBackoff = true


Is this correct? I don't get why it matters whether the error comes from the second+ stream.Recv() or the first one with respect to resetting backoff. Shouldn't it be io.EOF the conditional for a successful stream where we can infer the server is working properly?

dfawley · 2022-10-25T20:52:43Z

Ping @easwars @zasweq this should be ready to review again.

easwars

LGTM.

Deep down though, I would have still preferred if we could have added the methods required to make RPCs on the SubChannel interface. I feel that would have been a more intuitive change.

Like, currently, the RegisterOOBListener function in the orca package calls GetOrBuildProducer on the SubConn and passes a builder defined locally. This seems quite roundabout. But, if we want the orca package to handle everything locally, then we would need the functionality to make RPCs on the subConns, and also the functionality to uniquely identify subConns based on their channelz ID. If we don't think adding this functionality to the subConn interface is a good long term option, I agree with the approach taken in this PR.

orca/producer.go

zasweq

All testing nits.

orca/producer_test.go

zasweq · 2022-11-02T00:36:53Z

orca/producer_test.go

+	// Stop listener 3.  This makes the interval longer, with stream recreation
+	// delayed until the next report is received.  Reports only go to listener
+	// 1 now.


Reports should only go to listener 1 now. Also explain why the interval is now longer, and perhaps tie that idea of a longer interval to stream recreation.

ping about stream recreation.

I think a very high level understanding of the feature (ORCA OOB reporting) plus this comment as-is is sufficient to understand what's happening here. I could make a longer comment to make things painfully obvious, but sometimes less is more.

orca/producer_test.go

zasweq · 2022-11-02T22:00:39Z

orca/producer_test.go

+
+	lisOpts := orca.OOBListenerOptions{ReportInterval: 50 * time.Millisecond}
+	lao := &listenerAndOptions{listener: oobLis, opts: lisOpts}
+	r.InitialState(resolver.State{Addresses: []resolver.Address{setListenerAndOptions(resolver.Address{Addr: lis.Addr().String()}, lao)}})


I don't see this split up still.

zasweq · 2022-11-02T22:08:09Z

orca/producer_test.go

@@ -255,10 +258,14 @@ func (s) TestProducerBackoff(t *testing.T) {

 	// Provide a convenient way to expect backoff calls and return a minimal
 	// value.
-	expectedBackoff := -1 // -1 indicates any value is allowed.
+	const backoffShouldNotBeCalled = 9999 // Use to assert backoff function is not called.


Can you clarify why?

I do at the usage points: https://github.com/grpc/grpc-go/pull/5669/files#diff-6a71af69cb3159cbd1bbb20fb808da29d460ee437f134b485afc11df20079bb3R343-R345

I don't get that explanation, but it's probably just me.

I explain in comments why we don't want the backoff function to be called when backoffShouldNotBeCalled is assigned to expectedBackoff below. (It's because the last stream was successful.)

zasweq · 2022-11-02T22:08:34Z

orca/producer_test.go

+
+	// Register our fake ORCA service
+	s := grpc.NewServer()
+	fake := newFakeORCAService()


zasweq

Some comments.

zasweq · 2022-11-02T22:11:27Z

orca/producer_test.go

-	// Stop listener 2.  This does not affect the interval.  The next update
-	// goes to listeners 1 and 3.
+	// Stop listener 2.  This does not affect the interval as listener 3 is
+	// still the shortest.  The next update goes to listeners 1 and 3.


and still has the shortest interval

This level of review isn't important. The meaning of this comment is very clear, and it's an internal comment in a test, not in external documentation.

Wouldn't hurt though. But I see what you're saying

orca/producer_test.go

zasweq

LGTM outside my final responses, but again don't feel strongly about any of my nits.

zasweq · 2022-11-03T17:18:03Z

You can go ahead and merge btw.

… reports (grpc#5669)

dfawley added the Type: Feature label Sep 23, 2022

dfawley added this to the 1.50 Release milestone Sep 23, 2022

dfawley requested review from easwars and zasweq September 23, 2022 18:13

dfawley assigned easwars and zasweq Sep 23, 2022

dfawley force-pushed the orca_producer branch 3 times, most recently from 1b369f2 to 1cf3c60 Compare September 27, 2022 21:16

dfawley modified the milestones: 1.50 Release, 1.51 Release Oct 4, 2022

easwars reviewed Oct 5, 2022

View reviewed changes

easwars assigned dfawley and unassigned easwars Oct 5, 2022

easwars reviewed Oct 5, 2022

View reviewed changes

dfawley assigned easwars and unassigned dfawley Oct 6, 2022

easwars reviewed Oct 6, 2022

View reviewed changes

balancer/balancer.go Show resolved Hide resolved

balancer/balancer.go Outdated Show resolved Hide resolved

easwars assigned dfawley and unassigned easwars Oct 6, 2022

zasweq requested changes Oct 13, 2022

View reviewed changes

zasweq removed their assignment Oct 13, 2022

dfawley added 3 commits October 17, 2022 14:41

balancer: add Producer types and methods

ed0eb16

orca: create ORCA producer for LB policy use

e1c66fa

review comments

4971070

dfawley force-pushed the orca_producer branch from 1cf3c60 to 4971070 Compare October 17, 2022 21:42

dfawley commented Oct 17, 2022

View reviewed changes

zasweq reviewed Oct 18, 2022

View reviewed changes

dfawley added 2 commits October 18, 2022 09:51

don't log ctx.Err

31dd4a6

fix min computation

f6c6b4c

dfawley assigned easwars and zasweq and unassigned dfawley Oct 18, 2022

dfawley mentioned this pull request Oct 18, 2022

client: synchronously verify server preface in newClientTransport #5731

Merged

dfawley added 2 commits October 18, 2022 15:57

add grpcsync.OnceFunc

0961db3

use OnceFunc

b259326

easwars approved these changes Oct 27, 2022

View reviewed changes

orca/producer.go Show resolved Hide resolved

easwars assigned dfawley and unassigned easwars Oct 27, 2022

zasweq requested changes Nov 2, 2022

View reviewed changes

zasweq removed their assignment Nov 2, 2022

test updates

65653a4

zasweq reviewed Nov 2, 2022

View reviewed changes

orca/producer_test.go Outdated Show resolved Hide resolved

orca/producer_test.go Show resolved Hide resolved

zasweq reviewed Nov 2, 2022

View reviewed changes

zasweq requested changes Nov 2, 2022

View reviewed changes

more test changes

8aa79d6

dfawley assigned zasweq and unassigned dfawley Nov 2, 2022

zasweq approved these changes Nov 3, 2022

View reviewed changes

zasweq assigned dfawley and unassigned zasweq Nov 3, 2022

dfawley merged commit e41e894 into grpc:master Nov 3, 2022
1 check passed

dfawley deleted the orca_producer branch November 3, 2022 17:27

jronak pushed a commit to jronak/grpc-go that referenced this pull request Nov 21, 2022

orca: create ORCA producer for LB policies to use to receive OOB load…

faf2776

… reports (grpc#5669)

github-actions bot locked as resolved and limited conversation to collaborators May 3, 2023

orca: create ORCA producer for LB policies to use to receive OOB load reports #5669

orca: create ORCA producer for LB policies to use to receive OOB load reports #5669

Conversation

dfawley commented Sep 23, 2022

Choose a reason for hiding this comment

dfawley Oct 17, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

easwars left a comment

Choose a reason for hiding this comment

zasweq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dfawley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq Oct 18, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dfawley commented Oct 25, 2022

easwars left a comment

Choose a reason for hiding this comment

zasweq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq left a comment

Choose a reason for hiding this comment

zasweq commented Nov 3, 2022

dfawley Oct 17, 2022 •

edited

zasweq Oct 18, 2022 •

edited