REST API testing: enable use of non-standard gRPC servers that support gRPC transcoding for testing

[krousey]

Is your feature request related to a problem? Please describe.
It would be easier to test REST clients the same way I test gRPC clients if the generated gRPC service registration function accepted the grpc.ServiceRegistrar interface instead of a *grpc.Server concrete type.

There are non-standard gRPC servers¹ that support gRPC transcoding for the REST interfaces through protoreflection, but you have to be able to register the service with them. They implement the grpc.ServiceRegistrar interface, but they can't be used because the generated code doesn't use that interface. For example:

google-cloud-go/longrunning/autogen/longrunningpb/operations.pb.go

Lines 1108 to 1110 in ec9b526

	func RegisterOperationsServer(s *grpc.Server, srv OperationsServer) {
	s.RegisterService(&_Operations_serviceDesc, srv)
	}

Describe the solution you'd like
I would like you to update whatever protoc plugin used to generate this code to use the grpc.ServiceRegistrar interface. This would also match what the standard gRPC protoc plugin does:

https://github.com/grpc/grpc-go/blob/bb0d32f07806ec84fc7a7bfeff73a5c3b53555bb/examples/helloworld/helloworld/helloworld_grpc.pb.go#L89-L91

Describe alternatives you've considered

• Hand rolling all the REST mappings I need for my project...
• Running an external program that supported gRPC transcoding (envoy or https://github.com/grpc-ecosystem/grpc-httpjson-transcoding)
• generating mocks based on the client interface and using dependency injection for every client interaction
• using //go:linkname directives to expose the package private service descriptors so I could write the registration functions myself
  • These service descriptor variables are generally public already in most gRPC generated libraries.

Additional context
Most other gRPC generated services already support this.

Footnotes

1. https://github.com/emcfarlane/larking ↩

[codyoss]

This sounds like googleapis/go-genproto#1040. Which I think we do plan on taking action on in the new year. I am not sure if this would fix the issue though, but I may be misunderstanding your usecase. If you are using REST client for one of our services it does not make use of the gRPC tech stack, but more so standard HTTP and protojson to handle req/resp.

Although this is guide is for some of our older client libraries, the same principles could be used for REST clients in this repo as they share the same client options: https://github.com/googleapis/google-api-go-client/blob/main/testing.md#testing-http-services-using-fakes

[krousey]

@codyoss That would indeed address this issue. You can mark this a duplicate of that. The rest of this message is just explaining how this was related to testing REST apis for me.

The REST client testing is just my motivating reason for wanting this change. I know I can write my own HTTP handler and use httptest and option.WithEndpoint to point a client at the custom handler. That was the first bullet point in the considered alternatives section.

Instead of handrolling a bunch of REST handlers, I wanted to take advantage of the gRPC transcoding already defined in the proto files.

// The Networks API.
service Networks {

  // snip ...

  // Deletes the specified network.
  rpc Delete(DeleteNetworkRequest) returns (Operation) {
    option (google.api.http) = {
      delete: "/compute/v1/projects/{project}/global/networks/{network}"
    };
    option (google.api.method_signature) = "project,network";
    option (google.cloud.operation_service) = "GlobalOperations";
  }

  // Returns the specified network.
  rpc Get(GetNetworkRequest) returns (Network) {
    option (google.api.http) = {
      get: "/compute/v1/projects/{project}/global/networks/{network}"
    };
    option (google.api.method_signature) = "project,network";
  }

  // Creates a network in the specified project using the data included in the request.
  rpc Insert(InsertNetworkRequest) returns (Operation) {
    option (google.api.http) = {
      body: "network_resource"
      post: "/compute/v1/projects/{project}/global/networks"
    };
    option (google.api.method_signature) = "project,network_resource";
    option (google.cloud.operation_service) = "GlobalOperations";
  }

  // Retrieves the list of networks available to the specified project.
  rpc List(ListNetworksRequest) returns (NetworkList) {
    option (google.api.http) = {
      get: "/compute/v1/projects/{project}/global/networks"
    };
    option (google.api.method_signature) = "project";
  }

  // snip ...

}

I found a library ¹ that acts a gRPC server, but uses protoreflection on the registered descriptors to automatically map the REST api calls to gRPC calls based on the google.api.http options defined. So that let's me instead write something like

type fakeNetworksServer struct {
    computepb.UnimplementedNetworksServer
}

func (f *fakeNetworksServer) List(context.Context, computepb.ListNetoworksRequest) (*computepb.NetworkList, error) {
    return &computepb.NetworkList{}, nil
}

func TestNetworksList(t *testing.T) {
    l, _ := net.Listen("tcp", "localhost:0")
    
    // library I found
    mux, _ := larking.NewMux()
    srv, _ := larking.NewServer(mux)

    computepb.RegisterNetworksServer(src, &fakeNetworksServer{}) // This is what didn't work

    go func() {
        srv.Serve(l)
    }
    t.Cleanup(func() {
        srv.Close()
    }

    cl, _ := compute.NewNetworksRESTClient(context.Backgroun(),
        option.WithEndpoint("http://" + l.Addr()),
        options.WithoutAuthentication(),
    )

    cl.List(...)
}

This avoided having to discover/redefine all the REST mappings myself, and meant that I could test the REST client the exact same way I'm already testing gRPC clients as suggested in https://github.com/googleapis/google-cloud-go/blob/main/testing.md#testing-grpc-services-using-fakes

Footnotes

1. https://github.com/emcfarlane/larking ↩

[codyoss]

Closing as dupe of tagged issue