Skip to content

v1.21.0
Compare
Choose a tag to compare
@dsnet dsnet released this 13 Apr 19:12
v1.21.0
3b9eee1

Overview

This release contains some improvements to the usability of the protoc-gen-go tool and the release of several packages that make it easier to interact directly with the binary wire format.

Notable changes

New features:

  • CL/219298: compiler/protogen: add module= generator option
  • CL/219597: compiler/protogen: make paths=import work with M overrides
  • CL/223819: compiler/protogen: allow specifying the package name with M flags
  • CL/219838: encoding/protowire: make package publicly available
  • CL/219837: testing/protopack: make package publicly available
  • CL/221432: testing/protocmp: add SortRepeated and SortRepeatedFields
  • CL/226237: reflect/protodesc: add NewFiles

Minor changes:

  • CL/223817: types/dynamic: make Message implement legacy message interface
  • CL/223857: types/dynamicpb: fix message Zero return a read-only type

Code generator flags

A new module flag is added to the generator that controls stripping of a prefix from the generated filenames. This option enables protoc-gen-go to better generate .pb.go files for a workflow that depends on Go modules rather than a centralized Go path.

For example, suppose a protos/source.proto file has a go_package option specified as follows:

option go_proto = "github.com/example/project/foo";

and that the module root for this project is github.com/example/project.

If invoked from the module root, this flag could be potentially used as follows:

protoc --go_out=module=github.com/example/project:. protos/source.proto

This would emit a file located foo/source.pb.go rather than being located at github.com/example/project/foo/source.pb.go.

Furthermore, two minor changes have been made to the M flag, which can be used to specify the Go package path corresponding to a given .proto file:

  • When used with the paths=import option, the implementation now correctly respects the effect of that option and outputs the file as import-relative, rather than always being source-relative.
  • If the Go package path is provided with a ; delimiter, it can be used to specify the exact package name for the generated package. It's syntax is identical to that of the go_package option.

Package protowire

The protowire package provides functionality for parsing and packing the binary wire format. Effective use of this package requires that the user has an understanding of the wire format itself. The package is written in a performance sensitive way and currently serves as the underlying implementation of the protobuf runtime. It is the replacement for methods previously provided on the Buffer type in the legacy proto package.

Example of how to parse wire data for an encoded message:

import "google.golang.org/protobuf/encoding/protowire"

var b []byte = ... // encoded bytes of a protobuf message
for len(b) > 0 {
    num, typ, n := protowire.ConsumeField(b)
    if n < 0 {
        log.Fatalf("parse error: %v", protowire.ParseError(n))
    }
    log.Printf("FieldNumber:%d  WireType:%d  WireData:%x", num, typ, b[:n])
    b = b[n:]
}

Using the encoded output of the following protopack example, this produces:

FieldNumber:1  WireType:2  WireData:0a0d48656c6c6f2c20776f726c6421
FieldNumber:2  WireType:0  WireData:10f6ffffffffffffffff01
FieldNumber:3  WireType:2  WireData:1a0ccdcc8c3fcdcc0c4033335340

Package protopack

The protopack package is intended to be used for debugging the binary wire format or assist in the construction of testdata in the wire format. Effective use of this package requires that the user has an understanding of the wire format itself. The protopack package essentially provides a set of types that represents the wire data as a concrete syntax tree.

Given this proto3 message definition:

message MyMessage {
    string field1 = 1;
    int64 field2 = 2;
    repeated float32 field3 = 3;
}

Valid wire data for the message can be constructed as such:

import . "google.golang.org/protobuf/testing/protopack"

b := Message{
    Tag{1, BytesType}, String("Hello, world!"),
    Tag{2, VarintType}, Varint(-10),
    Tag{3, BytesType}, LengthPrefix{
        Float32(1.1), Float32(2.2), Float32(3.3),
    },
}.Marshal()

This results in the following output data:

0x0000  0a 0d 48 65 6c 6c 6f 2c  20 77 6f 72 6c 64 21 10  |..Hello, world!.|
0x0010  f6 ff ff ff ff ff ff ff  ff 01 1a 0c cd cc 8c 3f  |...............?|
0x0020  cd cc 0c 40 33 33 53 40                           |...@33S@|

Package protocmp

The SortRepeated and SortRepeatedFields options are added to the protocmp package. These options enable repeated fields to be sorted so that they are functionally treated as multisets.

The SortRepeated option sorts repeated fields of a specific element type. This option requires the user to provide a less function where the field type is inferred from the provided function type.

The SortRepeatedFields option sorts a set of repeated fields on a specific message type. This option requires the user to specify the set of fields by name. An arbitrary sort ordering is chosen for each specified repeated field.

Upcoming breakage changes

There are no new upcoming breaking changes to announce in this release.

As a reminder, the following breaking changes have already been announced and may take effect 6 months after they have first been announced:

Assets 8