Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Make grpc transcode logic work in terms of protobuf python objects #428

Merged
merged 4 commits into from Aug 30, 2022

Commits on Aug 26, 2022

  1. feat: Make grpc transcode logic work in terms of protobuf python objects

    (for context: [gRPC Transcoding](https://github.com/googleapis/googleapis/blob/master/google/api/http.proto#L44))
    Previously it worked on dictionaries only, but that causes problems.
    
    In GAPIC the dictionaries are created through the same logic as JSON (there is no better built-in way), thus applying [protobuf json mapping](https://developers.google.com/protocol-buffers/docs/proto3#json) conversion logic in the process. Unfortunately converting a protobuf object to a dictionary and to JSON, although similar, are not the same thing. Specifically the `Timestamp`, `Duration`, `FieldMask`, `uint64`, `int64`, and `*Value` protobuf messages are converted to strings for JSON (instead of being properly converted to dicts for most of those types, and `int64/uint64` converted to `int` respectively). As a result a rountrip that GAPIC was relying on (protobuf object -> dict -> transcode -> protobuf object) did not work properly. For example, when converted to dictionary, every int64 field would be converted to `string` (because it is what proto-JSON mapping spec requires), but later, when we need to rebuild a message from a transcoded dictionary that would fail with the following error:
    ```
    TypeError: '0' has type str, but expected one of: int
    ```
    
    Note, `*Rules` thing from proto-plus does not help, becuase the type may happen inside common native protobuf stub messsages (like `google.type.Money`), fields of which are outside of scope of the proto-plus custom conversion logic.
    
    Also, this change greatly simplifies the procedure of transcodding, eliminating multiple conversion steps (to and from dictionaries multiple times) making the whole logic significanly more efficient (python gapics are nutoriously known to be slow due to proto-plus stuff, so efficiency is important) and robust (JSON conversion logic does not interfere anymore with pure protobuf objects grpc transcoding)
    vam-google committed Aug 26, 2022
    Configuration menu
    Copy the full SHA
    cb7da88 View commit details
    Browse the repository at this point in the history
  2. reformat code using black

    vam-google committed Aug 26, 2022
    Configuration menu
    Copy the full SHA
    2876b64 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d81ed37 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2022

  1. Configuration menu
    Copy the full SHA
    b999840 View commit details
    Browse the repository at this point in the history