v0.13.0

This release includes a single addition to the protocompile.Compiler type.

What's Changed

Enhancements

• Provide ability for caller to seed/re-use symbol table across compile jobs by @jhump in #294

Full Changelog: v0.12.0...v0.13.0

v0.12.0

This release mostly contains performance improvements. It also includes an addition that makes it possible to implement a descriptor resolver on top of linker.Files much more efficiently, using a *linker.Symbols as an index.

What's Changed

Enhancements

• Add lookup methods to *linker.Symbols by @jhump in #286
• Various performance improvements (mostly reducing allocations) in the linker by @jhump in #286, #287, #290, and #291

Other changes

• Disallow options that use message-set-wire-format if Protobuf runtime can't support them by @jhump in #284
• Changes ast.NoSourceNode so that its methods now require a pointer receiver by @jhump in #291
  • This is a backwards-incompatible change. The impact should be low (likely even zero) since most users of this module will not be directly constructing ast.NoSourceNode values or type-asserting ast.Node values to ast.NoSourceNode. The change was made because of non-trivial performance gains, by greatly reducing the number of NoSourceNode values that need to be allocated on the heap when working with a parser.Result that has no AST.

Full Changelog: v0.11.0...v0.12.0

v0.11.0

This release includes some more Editions-related work.

What's Changed

Bugfixes

• Validate target type for all option fields, not just features by @jhump in #279
  • Previously, the targets field option was only validated for fields of google.protobuf.FeatureSet, but it is now validated for all fields set on option values.
• Fix issues with reporting of redundant/inaccurate errors by @jhump in #281

Enhancements

• Add Editions-related helper functions, for resolving features, to protoutil sub-package by @jhump in #283
• Allow access to original descriptor when wrapped in a linker.File via new Unwrap method by @jhump in #278

Other changes

• Proto2 group fields can use lower-case name in the text format (in option values that use message literals) by @jhump in #268
  • This aligns protocompile with a small change that is coming in protoc v27.0.

Full Changelog: v0.10.0...v0.11.0

v0.10.0

This release marks significant progress in implementing Protobuf Editions. Editions are not yet generally supported because there are a couple of remaining checks that need to be done. Until then, users can opt-in to using Editions source files with this compiler using the editionstesting.AllowEditions function.

This release also includes some backwards-incompatible changes, though they likely will not have any impact on most users. They are changes to interfaces that are compatible for users/callers of the interface, but incompatible for any code outside of this repo that implemented those interfaces.

What's Changed

Bugfixes

• In the recently added options.StripSourceRetentionOptionsFromFile, clear options if no fields remain and also clear source code info, in #251
• This repo previously allowed some sources that protoc would reject, due to incomplete validation of options. These checks were added in #264:
  • Non-lite files may not import lite files. (A lite file is one that uses option optimize_for=LITE_RUNTIME.)
  • Extensions in a lite file may not extend messages in a non-lite file.
  • Field options lazy and unverified_lazy can only be set to true for message fields.
  • Field option jstype can only be customized for 64-bit integer fields (int64, uint64, sint64, fixed64, and sfixed64).

Enhancements

• Editions-related updated:
  • Update descriptor implementations to work with Editions files, in #260, #261, and #270
  • Add more internal compiler support for Editions, in #264
  • Provide an exported function for users to opt-in to using Editions, in #267. Note that this capability is temporary and will be removed once Editions support is complete (since opt-in won't be necessary and Editions support will be generally available).
• Support and enforce extension declarations, in #263

Other Changes

• Remove feature related to "canonical bytes", in #261
  • This is a backwards-incompatible change, but only impacts a likely-unused feature. The method linker.Result.CanonicalProto() has been removed. This feature was too complicated to continue supporting in the face of changes to interpreting options that needed to be implemented for Editions. (The feature was ultimately never used for its original intended purpose, so is likely not used at all.)
• Add support for iterating option nodes in the ast sub-package via new ast.NodeWithOptions interface, in #263
  • This is a backwards-incompatible change, but only impacts code that implements some of this repo's interfaces, not any code that merely uses those interfaces.
  • This adds a new RangeOptions function to several sub-interfaces of ast.Node, so they also are now sub-interfaces of ast.NodeWtihOptions. The interfaces changed are ast.FileDeclNode, ast.MessageDeclNode, ast.FieldDeclNode, ast.OneofDeclNode, ast.RPCDeclNode, and ast.EnumValueDeclNode.
  • This also adds a method, ExtensionsNode, to the parser.Result interface, and expands the return types of three methods (OneofNode, EnumNode, and ServiceNode) from ast.Node to ast.NodeWithOptions.

Full Changelog: v0.9.0...v0.10.0

v0.9.0

This release contains a minor change and one new exported function.

Improvements:

• The experimental github.com/bufbuild/protocompile/parser/fastscan package has been updated so it's even faster: it can scan in roughly half the time using a fourth as much memory as the previous version.

Additions:

• A new options.StripSourceRetentionOptionsFromFile helper function has been added, for use by tools that use the results of this compiler to do code generation. This function accepts a file descriptor and returns a modified version of it that has all options marked as "source" retention removed.

v0.8.0

This release primarily fixes bugs in the compiler that resulted in differences between the source files that protocompile would accept vs. the source files that the reference compiler, protoc, would accept. But one of these changes resulted in an unused AST node type. So this type has been removed from the exported API, which is why this v0.8.0 instead of v0.7.2.

Fixes:

1. A protocompile.Compiler would previously accept some sources that would be rejected by protoc. In particular, protocompile allowed for explicitly-signed positive numeric literals -- with a leading plus (+) sign. But this is not allowed by the reference compiler, protoc, nor by implementations of the text format in myriad languages (which is also the format the compiler uses to parse custom option values that are message literals).
   • This change is not backwards compatible. The AST node type ast.PositiveUintLiteralNode has been removed. It was only used to support this invalid construct in the language. Code that refers to this node type (or its associated factory function and Visitor method) must be removed.
2. A protocompile.Compiler would previously reject some sources that would protoc would accept. In particular, protocompile did not allow -nan as a default field value or custom option value and it also did not accept inf and alternate case spellings (such as upper- or mixed-case) of infinity and nan in message literals, which are all allowed by protoc.

v0.7.1

Fixes:

1. If a protocompile.Compiler's Resolver returns a *descriptorpb.FileDescriptorProto (instead of source code or an already-built protoreflect.FileDescriptor), any already-interpreted options therein would be inadvertently cleared when the compiler reached the step of interpreting options for that file. This has been fixed: descriptor protos with options already interpreted will be left alone.

v0.7.0

This release contains some potentially breaking changes. Such changes were deemed acceptable since this module has not yet reached v1.0 and the impact on existing user code is likely to be very low.

Fixes:

1. Previously, protocompile would allow the packed field option to be used on non-repeated fields are repeated fields that cannot be packed (like repeated string, bytes, and message fields). Such sources would be rejected by the reference compiler, protoc, and the resulting descriptors would be rejected by some Protobuf runtime libraries (including the one for Go). This has been fixed. So protocompile now also rejects such incorrect sources.
2. A panic could occur in the parser with certain kinds of malformed input that contained syntactically incorrect message literals in option values. This has been fixed, and such sources now result in a syntax error value instead of a panic.

Changes:

1. The ErrorWithPos interface has two new methods, Start() ast.SourcePos and End() ast.SourcePos. This change allows errors to indicate an entire region of code, not just a single position.
   • This is a breaking change. It is expected that this interface was only implemented internally. If there are implementations outside of this repo, fixing them can be as straight-forward as implementing the two new methods to return the same position as the existing GetPosition() method.
2. The various helper functions for creating ErrorWithPos instances now accept an ast.SourceSpan instead of an ast.SourcePos. This includes reporter.Error and reporter.Errorf as well as the methods HandleErrorWithPos, HandleErrorf, HandleWarningWithPos, and HandleWarningf of reporter.Handler.
   • This is a breaking change. It is expected that these functions, despite being exported, were only used internally by the compiler implementation. If there are existing calls outside of this repo and a span (such as a ast.NodeInfo, ast.ItemInfo, or ast.Comment) is not readily available to them, they can use ast.NewSourceSpan to construct a span whose start and end are the same ast.SourcePos and pass that.

Additions:

1. A new ast.SourceSpan interface type has been added that represents a span of code. There are also two factory functions for creating instances of ast.SourceSpan: ast.NewSourceSpan and ast.UnknownSpan. This interface is implemented by existing types ast.NodeInfo, ast.ItemInfo, ast.Comment, and reporter.ErrorWithPos.
2. A new github.com/bufbuild/protocompile/parser/fastscan package has been added. While this entire repo is pre-v1.0, and therefore may be subject to incompatible changes (as evidenced by the backwards-incompatible change in this release), this new package is considered experimental. So the threshold for allowing an incompatible change to this package is even lower, and its API should be viewed as less stable.

v0.6.0

This release contains several breaking changes. Such changes were deemed acceptable since this module has not yet reached v1.0 and the impact on existing user code is likely to be very low.

Fixes:

1. If a protocompile.Resolver returns search results that include descriptor protos, the compilation step that interprets options could inexplicably fail. This was because the logic was relying on an AST for some metadata about the option declarations in the source. This has been fixed: if a descriptor proto is provided that contains uninterpreted options, they can now be successfully interpreted.

Changes:

1. Numerous elements have been renamed so that references to "oneof" elements now use the spelling "Oneof" instead of "OneOf" (second "O" now lower-case). This is for consistency with how this is spelled in the core protobuf runtime, in the "google.golang.org/protobuf" module).
   • This is a breaking change. Fixes to user code that may have been broken by this change should be straight-forward and mechanical.
2. The grammar used to generate the parser has been improved. In particular, all conflicts in the yacc configuration have been eliminated. This does not fix any known bug, but it does provide higher confidence that no latent bugs exist since the generated parser could previously have chosen the wrong action in the face of a conflict.
3. The function sourceinfo.GenerateSourceInfo now accepts a sourceinfo.OptionIndex instead of an options.Index. It also now accepts variadic options. The function sourceinfo.GenerateSourceInfoWithExtraComments has been removed; its functionality can now be accomplished by passing an option to sourceinfo.GenerateSourceInfo.
   • This is a breaking change. It is expected that these functions, despite being exported, were only used internally by the compiler implementation.
4. The type options.Index has been removed. Code should instead use the new sourceinfo.OptionIndex type.
   • This is a breaking change. It is expected that this type, despite being exported, was only used internally by the compiler implementation.

Additions:

1. A new protocompile.SourceInfoMode value has been added: SourceInfoExtraOptionLocations. This can be combined with other values (via bitwise OR) to have the compiler include additional source code info for locations inside message literals in custom option values.
2. A new sourceinfo.OptionIndex type has been added, along with related types in the same package: OptionSourceInfo, OptionChildrenSourceInfo, ArrayLiteralSourceInfo, and MessageLiteralSourceInfo. These provide a model of source info paths in the interior of an option declaration.
3. A new sourceinfo.GenerateOption type has been added, allowing the behavior of sourceinfo.GenerateSourceInfo to be customized via variadic options. Options are available for retaining extra comments (sourceinfo.WithExtraComments) and for producing extra locations for option declarations with message literal values (sourceinfo.WithExtraOptionLocations).

v0.5.1

This is an urgent bug-fix release, to remedy a potential deadlock error in the previous release (v0.5.0).

The only change in this release is a fix to this deadlock issue. With v0.5.0, if a compile operation was using an override version of google/protobuf/descriptor.proto, it was possible for the call to compiler.Compile to halt, only returning if the context was cancelled or times out. This is now fixed.