Releases: jhump/protoreflect
v1.14.1
v1.14.0
"github.com/jhump/protoreflect/dynamic"
Changes/fixes:
- If converting or merging a generated message into a
*dynamic.Message
, if the generated message contained dynamic extensions (extension fields not known to the Protobuf runtime's global registry), they would be lost during the conversion. This has been fixed.
"github.com/jhump/protoreflect/grpcreflect"
Additions:
- Adds a new
grpcreflect.NewClientAuto
method, which returns a client that will automatically use either the v1 version of server reflection or v1alpha, depending on what the server supports. - Adds a new
grpcreflect.NewClientAlpha
method which is meant to replacegrpcreflect.NewClient
. It always uses the v1alpha version of server reflection (and its predecessor,grpcreflect.NewClient
, is now deprecated since it is unclearly named).
v1.13.0
This version includes mostly bug fixes. It does include some new API and features in desc/builder
and desc/sourceinfo
packages. This version also introduces support for Windows. (Some packages did not work correctly on platforms where the default path separator was not /
. These packages have been fixed.)
"github.com/jhump/protoreflect/desc"
Changes/fixes:
- If using the
CreateFileDescriptor
functions to convert descriptor protos into "rich" descriptors, a panic could occur on certain kinds of invalid input. For example, if the input type of an RPC method referred to an enum instead of a message, a type assertion panic would be triggered. This has been fixed so that such incorrect input results in the function returning an error, not panic'ing.
"github.com/jhump/protoreflect/desc/builder"
Additions:
- Adds a new method
PruneUnusedDependencies
to theFileBuilder
type. This can be used to remove any unused dependencies. This is mainly useful when modifying an existing descriptor and removing elements from it. After removing elements, some of the file's imports may no longer be used.
"github.com/jhump/protoreflect/desc/protoparse"
Changes/fixes:
- Fixes a potential panic when parsing and linking source files. If the source contains a symbol that conflicted with the name of a synthetic oneof (an implementation detail of how descriptors with "proto3 optional" fields are generated), it would trigger a nil de-reference panic while trying to construct the error message. This has been fixed.
- Fixes a subtle issue when resolving relative references in field types and extendees, whereby this package would previously accept a reference that
protoc
would reject. The resolution rules inprotoc
handle partially-qualified references a little differently than unqualified references, and this package didn't previously implement the subtle difference. This is now fixed. - Fixes an issue where a field type was allowed to directly refer to the name of a synthetic map entry message. This is not actually allowed by
protoc
. Now such an incorrect usage results in an appropriate error. - Adds checks to verify that default JSON names for fields and enum values do not conflict. This implements the check in the same fashion as
protoc
, which is case-insensitive. For enum values, any prefix on the value name that matches the enum name (regardless of case or underscores) is disregarded when performing this check. The check results in an error for proto3 syntax files, but only a warning for proto2 syntax files. - Adds a check to make sure that custom
json_name
options on fields are valid. So if two fields try to declare the samejson_name
, this will be considered an error. For proto3 syntax files, if a customjson_name
option conflicts with another field's default JSON name, this is also an error (but only a warning for proto2 syntax files). - The
ParseFilesButDoNotLink
method will now return uninterpreted options in a form that matches the way the official C++Parser
class does. In particular, aggregate values now use the same string format, which will be identical to the original message literal in source but without the enclosing braces ({
and}
), with all comments and whitespace removed, and with a single space inserted between lexical elements.
"github.com/jhump/protoreflect/desc/protoprint"
Changes/fixes:
- Previously, characters outside US ASCII (7 bit) in string literals would be encoded using octal escapes when printed. (There is not enough information in a descriptor to preserve the original source's encoding.) This made string literals with other unicode code points (other languages/alphabets, emoji, etc) impossible to read. This has been fixed. String literals used for
bytes
fields still use aggressive octal escapes. But forstring
fields, escapes are only used for non-printable code points.
"github.com/jhump/protoreflect/dynamic"
Changes/fixes:
- The
GetKnownType
method ofKnownTypeRegistry
could previously return an invalid type, if given the fully-qualified name of a synthetic map entry message. It now returnsnil
in such a case.
"github.com/jhump/protoreflect/desc/sourceinfo"
Additions:
- This release adds a new
RegisterEncodedSourceInfo
function, used by newer code generated byprotoc-gen-gosrcinfo
. It accepts compressed encoded data in the form of a byte array.
"github.com/jhump/protoreflect/desc/sourceinfo/cmd/protoc-gen-gosrcinfo"
Changes/fixes:
- The generated code now emits compressed information. That way, it uses less space in the executable, resulting in smaller binaries.
v1.12.0
This change includes a new feature related to source code information for compiled-in proto descriptors. It also includes some other bug fixes.
"github.com/jhump/protoreflect/desc"
Changes/fixes:
- The
FindSymbol
method ofFileDescriptor
would panic if given an empty string as the symbol name. This has been fixed. This case now returns a nil descriptor, indicating no element with the given name could be found. - The various
Load*
functions will return descriptors that include comment information (and the underlyingFileDescriptorProto
will contain source code info) if theprotoc-gen-gosrcinfo
plugin was used to generate additional Go code for these proto files.
"github.com/jhump/protoreflect/desc/protoprint"
Changes/fixes:
- If an option included a message type, and was printed as a compact message literal, it had an errant extra space before the final closing '}' symbol. This is fixed, and these are now printed with a single space between the last field value and the closing brace.
"github.com/jhump/protoreflect/desc/sourceinfo"
This is a brand new package which allows for registering source code information for a proto source file. It also includes some API for using/querying the source code info. This is useful to retain comments for compiled and linked-in descriptors. It is not expected that users would explicitly register source code info; instead use the protoc-gen-gosrcinfo
plugin, in addition to protoc-gen-go
, when compiling your proto sources.
"github.com/jhump/protoreflect/desc/sourceinfo/cmd/protoc-gen-gosrcinfo"
This is a brand new command which is a protoc
plugin for augmenting compiled and linked-in descriptors with source code information. When used, the desc
package can include source code info (and thus comments) when loading linked-in descriptors. The desc/sourceinfo
package can be used to query for the source code info and can be useful for integrating this data with other things, such as gRPC server reflection.
v1.11.0
"github.com/jhump/protoreflect/desc/protoprint"
Changes/fixes:
- If an option on a method included comments and was the kind of option that
protoprint
is able to handle, they would fail to be included in the printed output. Options were supported on all other types, just not on methods. This has been remedied: methods now have the same support for options comments as other elements. - The
Printer
type includes three new fields, to control formatting of complex options:ShortOptionsExpansionThresholdCount
andShortOptionsExpansionThresholdLength
control when "short options" will be expanded to multiple lines, based on the number of options and the length of the options when rendered;MessageLiteralExpansionThresholdLength
controls when message literals in option values will be expanded to multi-line form, based on if the length of the rendered string is too long.
"github.com/jhump/protoreflect/grpcreflect"
Changes/fixes:
- The
LoadServiceDescriptors
function now accepts an interface, not just the concrete type*grpc.Server
. Since*grpc.Server
implements the interface, this change should be mostly backwards compatible. However, if there are usages that rely on the precise signature, such as assigning to a function variable whose signature requires*grpc.Server
, this is a minor breaking change. Such usage is not expected.
v1.10.3
This release contains several fixes to the desc/protoparse
package and one fix to the grpcreflect
package.
"github.com/jhump/protoreflect/desc/protoparse"
Changes/fixes:
- If a custom option value contains a message literal,
protoc
accepts identifierst
,f
,True
, andFalse
as synonyms fortrue
andfalse
. Use of these alternate spellings would be rejected by this package. This is now fixed so that this package accepts the same values asprotoc
. - If a custom option refers to an enum which has values named
true
orfalse
, this package would reject the option, misunderstanding thetrue
orfalse
identifier to be a boolean literal instead of an enum value name. This has been fixed. - There were several cases where references to a custom option or extension, in an option name or in a message literal, were resolved differently by this package than by
protoc
. This lead to some variances -- some source files thatprotoc
would accept but that this package would not (because it was unable to resolve a reference), and some source files that this package would accept but thatprotoc
would reject (because this package was using different lexical scoping rules during resolution). This has been fixed and this package now resolves extension names the same way asprotoc
. - When specifying a custom option value for a message whose type is
google.protobuf.Any
,protoc
supports a special syntax that makes it possible to use a simple text format for the contained message, instead of having to include a byte string representation of a marshaled/encoded value. This package did not previously implement that syntax, so would reject proto sources that used it. This has been remedied, and the special syntax is now supported by this package. - This package would previously accept a
repeated
extension for a message that used message-set wire format. However, onlyoptional
extensions are allowed for such messages. This has been fixed, and proto sources that try to define such arepeated
extension will be rejected. - Extensions are not allowed to set a
json_name
option, however this package was accepting proto sources that did so. This has been fixed, and proto sources that define an extension with thejson_name
option will be rejected.
"github.com/jhump/protoreflect/grpcreflect"
Changes/fixes:
- In some cases, servers implementing the reflection service has been observed to incorrectly include extra file descriptors in response to a
file_containing_symbol
request. Also, the reflection service does not actually specify any ordering requirements for responses that choose to include more than one file. But this package mistakenly assumed an ordering (based on an older implementation of the reflection service in the official Java runtime), which could cause such cases (responses with multiple or even superfluous files) to return the incorrect file descriptor. This has been fixed. Now all responses tofile_containing_symbol
,file_containing_extension
andfile_by_filename
requests correctly support multiple files (even superfluous ones) in any order.
v1.10.2
This release contains several fixes to the desc/protoparse
and dynamic
packages.
"github.com/jhump/protoreflect/desc/protoparse"
Changes/fixes:
- If a custom option has a type with a
oneof
in it, then option values in proto sources should not be allowed to set more than one of the fields. Previously, this package would accept such sources, and only the last field (in source location order) would end up being set in the option value. Now the package will report an error. - This package would incorrectly accept numeric literals in formats that the Go language accepts but that the Protocol Buffer does not. For example: "0b0100", "0o1735", and "1_000_000" are not legal integer literals in proto source, but are legal Go integer literals and were incorrectly accepted by this package. Similarly, Go accepts "1_000.314" as a float literal and so did this package. But this is not allowed in proto source. These issues have been fixed and this package now accepts numeric literals using the same rules as
protoc
. - This package would report an error if a numeric literal over-flowed what can be represented by a 64-bit floating point value (IEEE754 double precision). However,
protoc
accepts such sources and, like other floating point literals that cannot be perfectly represented, picks the nearest/best option. In the case of overflow, that value is infinity. This package now behaves likeprotoc
and substitutes an infinite value for numeric literals that would otherwise overflow a 64-bit floating point value. - A
NUL
character in a source file (code point zero,0x00
) could be misinterpreted as end-of-file. This character is not allowed in proto sources, but would not only be accepted in some cases but also result in incomplete compilation. Similarly, this package accepted such characters inside of comments, but that is also not allowed. This has been remedied and source files with such characters will be rejected. - This package would incorrectly allow an integer literal like "09", treating it as decimal. However, in proto sources, an integer literal that starts with a zero must be octal (in the given example, "9" is out of range and should have resulted in error). This is fixed.
- This package would allow reserved ranges and extension ranges to start at zero. However one is lowest acceptable value for such ranges. This has been fixed.
- This package would incorrectly allow two different oneofs in the same message to have the same name. This has been fixed.
- This package would allow proto source files that try to directly import the same file more than once. While it is allowed for the transitive dependency graph of a file to include a file more than once, a single proto source file cannot reference the same file multiple times. This is fixed and now reported as an error.
- Custom option values that used the message literal syntax were required to use a colon separator between a repeated field name and an array literal (e.g. multiple values inside of
[
and]
). However,protoc
does not require the colon in this context. This package has been updated to match the behavior ofprotoc
. - Parsing a source file could lead to a runtime panic under some conditions of symbol collisions. For example, if an operation used a built-in/well-known import (such as
google/protobuf/descriptor.proto
) without supplying its own copy of source and then included a source file that defined a symbol with the same fully-qualified name as an element in that import, a panic would occur. This has been fixed and the conflict is reported as an error as expected. - This package previously populated trailing comments differently from
protoc
for block elements (i.e. those with elements defined inside of brances{ }
, including messages, groups, oneofs, extend blocks, enums, services, and methods). This package used the comment immediately following the closing brace (}
) as the element's trailing comment. However,protoc
uses the comment immediately following the opening brace ({
) as the trailing comment. The behavior in this package has been changed to match that ofprotoc
.
"github.com/jhump/protoreflect/desc/protoprint"
Changes/fixes:
- This package previously assumed that trailing comments for block elements (i.e. those with elements defined inside of brances
{ }
, including messages, groups, oneofs, extend blocks, enums, services, and methods) should be rendered after the closing brace. However, this is not how trailing comments are parsed byprotoc
: they are the comment immediately following the opening brace instead. So this package has been updated to render the trailing comment for these elements after the opening brace.
"github.com/jhump/protoreflect/dynamic"
Changes/fixes:
- When unmarshaling JSON into a dynamic message, where one of the fields is of type
google.protobuf.Value
, this package would incorrectly process values that were JSON arrays. This should be allowed and result in the field'slist_value
field being set with the contents of the array. Instead, the field would be set to a single value equal to the last value in the JSON array. This has been fixed. - When marshaling a dynamic message to JSON, if a field used a custom JSON name that included a character that was non-printable or outside of the 7-bit range (e.g. a control character, code points 0-31, or any code point >= 127), the result JSON output would be invalid. The characters would be encoded in a way such that the resulting key was not a valid JSON string. This has been fixed.
v1.10.1
This release contains some fixes to the protoparse
package.
"github.com/jhump/protoreflect/desc/protoparse"
Changes/fixes:
- Use of "groups" (an old feature only supported in syntax proto2) in custom options did not work correctly. The parser would incorrectly reject use of groups in some cases. And when using message literals as option values, it did not accept the same input as
protoc
with regards to the field name used to refer to a group value. These issues have been corrected and behavior is not consistent withprotoc
.
v1.10.0
This release contains some improvements to the protoparse
package and new functionality in the protoprint
package.
"github.com/jhump/protoreflect/desc/protoparse"
Changes/fixes:
- A couple of cases were identified where the
protoc
compiler would reject proto source, but theprotoparse
package would accept it:- The way
protoc
resolved relative names to fully-qualified names, for the request and response types in a method signature, differed slightly fromprotoparse
. Theprotoparse
package might accept a source program, butprotoc
might reject it due resolving a type name to something other than a type (such as a service or method name). This package now matches the behavior ofprotoc
and will reject the same kinds of source. - The
protoparse
would accept a program with a range like1to max
, for example, whereasprotoc
requires a space between the1
and the keywordto
. This was due to a small difference in how tokens are lexed. The lexer in this package has been updated to now match the behavior ofprotoc
and will now issue an error if there is no space between a numeric literal and subsequent identifier/keyword.
- The way
- The
protoc
compiler accepts'\v'
(vertical tab) and'\f'
(form feed) characters as whitespace, butprotoparse
would not. This is now fixed. - The
protoparse
package could panic (with type assertion failures or with index-out-of-range failures) if the input contained a unicode code point in a particular range (well outside the range of allowed characters in a proto source file, other than inside string literals). This has been fixed.
"github.com/jhump/protoreflect/desc/protoprint"
Additions:
- A new
CustomSortFunction
field has been added to thePrinter
type. This can be used to define a custom order in which elements are printed to a file. This also includes a new interface type,protoprint.Element
, which custom sort functions can use to inspect the elements for deciding how to order them.
v1.9.0
This release contains numerous improvements to the protoparse
package, to more closely match protoc
in terms of proto source files that are acceptable. It also contains some fixes in other packages.
"github.com/jhump/protoreflect/desc/builder"
Changes/fixes:
- When adding a message to another (to make a nested/enclosed type), the target enclosing message could be incorrectly detached from its parent element. This was the result of a typo in the implementation code and has been fixed.
"github.com/jhump/protoreflect/desc/protoparse"
Additions:
- The
protoparse
package now issues warnings when it detects that a source file has unused imports. This mirrors the warnings thatprotoc
issues in the same cases. This feature requires the use of aWarningReporter
with a parse operation. The concrete type of value provided to the warning reporter will be aprotoparse.ErrorUnusedImport
.
Changes/fixes:
- The
protoc
compiler was more strict thanprotoparse
when it comes to resolving relative (vs. full qualified) names. This led to conditions whereprotoparse
would accept a proto source file thatprotoc
would reject. The issue is when the first component of an identifier could match multiple lexical scopes. In such a caseprotoc
only matches the most enclosing scope. Butprotoparse
would fallback to other enclosing scopes if the most enclosing scope could not be used to resolve a symbol. (Hard to describe succinctly, so see the example in this bug report.) This issue is now fixed andprotoparse
resolves names in the same manner asprotoc
. - The
protoc
compiler uses "C++ enum scoping rules" for protobuf enums. This means that enum values are declared in the namespace of the enclosing enum (as siblings of the enum itself). Butprotoparse
incorrectly treated the enum as the parent scope/namespace. This led to source files thatprotoparse
would accept but thatprotoc
would reject. This issue is now fixed. - The use of custom options in
oneof
statements could incorrectly result in error messages about failing to resolve the custom option name, even if the source file and the option reference were valid. This has been fixed.
"github.com/jhump/protoreflect/dynamic/msgregistry"
Additions:
- A new error type,
ErrUnexpectedType
, was introduced. When a call toFindMessageTypeByUrl
orFindEnumTypeByUrl
fails because of a type mismatch (expecting a message, got an enum, or vice versa), this can now be determined programmatically by type-asserting the error to the new error type. This provides a proper/robust way to detect this kind of error (previously, callers would have to examine the error text, which is quite brittle).