Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify check for Google::Protobuf::Map type - redux #1541

Merged
merged 3 commits into from
Jun 23, 2023

Conversation

paracycle
Copy link
Member

Motivation

Fix #1533

Implementation

The realization is that you can't assign an array type to Map fields and that you can't assign a hash type to Repeated fields. So we can use that to detect the type of the field.

We try to instantiate a new instance of the constant type, setting the value of the current descriptor field to an empty hash. If the type is Map, then it will succeed. If it's Repeated, it will fail.

Another small change in this PR makes it so that when we fail to read the type of the descriptor for repeated/map fields, we won't end up generating invalid RBI files, due to the message class not having a name.

Tests

Added failing tests that pass after the changes.

@paracycle paracycle requested a review from a team as a code owner June 22, 2023 21:54
@paracycle paracycle force-pushed the uk-protobuf-fix-again branch 2 times, most recently from e1dbe63 to 6e60346 Compare June 22, 2023 22:01
Copy link
Contributor

@dirceu dirceu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! Thank you very much <3

I've tested this against Shopify's core monolith, and I can see 0 changes in compiled DSL RBIs (and consequently, no typechecking failures).

@@ -78,7 +78,7 @@ class Field < T::Struct

extend T::Sig

ConstantType = type_member { { fixed: Module } }
ConstantType = type_member { { fixed: T::Class[T.anything] } }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, could this cause problems on applications using a Sorbet version older than sorbet/sorbet#6781?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bitwise-aiden
Copy link
Contributor

Another thing to add would be that the Ruby protobuf implementation has an open PR to expose the options field on descriptors. Once that has been released we should be able to switch to querying that directly.

protocolbuffers/protobuf#12828

@@ -206,7 +206,7 @@ def type_of(descriptor)
# > field, even if it was not defined in the enum.
"T.any(Symbol, Integer)"
when :message
descriptor.subtype.msgclass.name
descriptor.subtype.msgclass.name || "T.untyped"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a name in the submsg_name. I think that we should add that before the T.untyped.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that returns the name of the protobuf message, not the class name; I was wrong about that in the previous change.

For example, in the binary serialized test, if we hadn't assigned the MyCart.Progress message to a constant, its subtype.msgclass.name would be nil, but it would return MyCart.Progress from submsg_name, which is not a valid constant name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right! Would MyCart::Progress be a valid constant in that case though?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it wouldn't since there is no named constant defined in Ruby that corresponds to that msgclass. That constant is anonymous, so we can't refer to it.

If the `msgclass` of a subtype is not assigned to a constant, then its `#name` method would return `nil` and the compiler would return it as the type name of the descriptor. Not only does this break the contract of the `type_of` method defined by its signature which promises to always return a `String` instance, but it also leads to us generating broken RBI files, since these `nil` values serialize to empty strings.

 This commit fixes this by mapping the message class to `T.untyped` if it does not have a name.
The realization is that you can't assign an array type to `Map` fields and that you can't assign a hash type to `Repeated` fields. So we can use that to detect the type of the field.

We try to instantiate a new instance of the constant type, setting the value of the current descriptor field to an empty hash. If the type is `Map`, then it will succeed. If it's `Repeated`, it will fail.
@paracycle paracycle merged commit d08c4b1 into main Jun 23, 2023
29 checks passed
@paracycle paracycle deleted the uk-protobuf-fix-again branch June 23, 2023 17:54
@shopify-shipit shopify-shipit bot temporarily deployed to production June 23, 2023 18:32 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Google::Protobuf DSL generation broken again
3 participants