The runtime library for the generated code is provided by the npm package @bufbuild/protobuf. This is a detailed overview of the features provided by the library.
- Message class
- Enumerations
- Extensions
- Well-known types
- 64-bit-integral-types
- Reflection
- Advanced serialization
- Advanced TypeScript types
All generated messages extend the base class Message.
It provides a few helpful methods to compare, clone, and serialize, and a convenient constructor.
All message classes also come with some static properties with metadata, and static convenience methods.
For the following examples, we will use the following message definition example.proto:
syntax="proto3";
package docs;
message User {
string first_name = 1;
string last_name = 2;
bool active = 3;
User manager = 4;
repeated string locations = 5;
map<string, string> projects = 6;
}You can create an instance with the new keyword:
const user = new User();For convenience, constructors accept an initializer object. All fields in the initializer object are optional, and if not provided, the default value for the field is used.
const user = new User({
firstName: "Homer",
active: true,
manager: { // you can simply pass an initializer object for this message field
lastName: "Burns",
},
});The initializer object accepted by all message constructors is of type
PartialMessage<T> where T is your message type. So
in the above example, the initializer object is of type PartialMessage<User>.
PartialMessage is similar to the TypeScript built-in type Partial, but works
recursively. For more details, see the below section on Advanced TypeScript types.
If you need to define the initializer object independent of the constructor,
then be sure to use a type assertion, otherwise you may see unexpected compile
errors with oneof fields. In TypeScript 4.9 and above, it is recommended to
use satisfies.
const obj = {
firstName: "Homer",
active: true,
manager: {
lastName: "Burns",
},
} satisfies PartialMessage<User>;
const user = new User(obj);If you are not using TypeScript 4.9 yet, use as const instead of satisfies PartialMessage<User>.
Fields are automatically set to their default value if you create a message with
the new keyword. You can see that in the TypeScript generated code, where fields
with a default value are class properties with a default value:
/**
* @generated from field: string firstName = 1;
*/
firstName = "";Protobuf fields map to default values as follows:
| Protobuf field | Class property default value |
|---|---|
| bool | false |
| string | "" |
| other scalar types | see scalar field default values |
| optional scalar | undefined |
| message | undefined |
| map | {} |
| repeated | [] |
| oneof | { case: undefined } |
Fields translate to plain properties. You can set and get field values with simple property access:
user.firstName = "Homer";
user.manager = new User();
user.firstName; // "Homer"
user.manager?.active; // false You can also use a destructuring assignment:
let {firstName, lastName} = user;The same is true for repeated and map fields, but fields in a oneof group are
grouped into an object property.
With the following oneof group:
message User {
string first_name = 1;
string last_name = 2;
bool active = 3;
User manager = 4;
repeated string locations = 5;
map<string, string> projects = 6;
+ oneof result {
+ int32 number = 4;
+ string error = 5;
+ }
}A property named result (the name of the oneof group) is added to the message.
It is an object with two properties:
case- the name of the selected fieldvalue- the value of the selected field
This property is always defined on the message - similar how map or repeated
fields are always defined. By default, it is {case: undefined}.
In our example, result.case can be either "number", "error", or undefined.
If a field is selected, the property result.value contains the value of the
selected field.
In TypeScript, these rules are actually enforced by the declaration:
result:
| { case: "number"; value: number }
| { case: "error"; value: string }
| { case: undefined; value?: undefined } = { case: undefined };To select a field, simply replace the result object:
user.result = {case: "number", value: 123};
user.result = {case: undefined};To query a oneof group, you can use if-blocks:
if (user.result.case === "number") {
user.result.value; // a number
}Or a switch statement:
switch (user.result.case) {
case "number":
user.result.value; // a number
break;
case "error":
user.result.value; // a string
break;
}This representation is particularly useful in TypeScript, because the compiler
narrows down the type. That means the if blocks and switch statements above tell
the compiler the type of the value property. Note that type narrowing requires
the TypeScript compiler option strictNullChecks.
This option is automatically enabled with the option strict, which is recommended.
While a shallow copy of a message can be created by using the spread operator with the message constructor, it is also possible to create a deep clone of a message:
user.clone();We provide instance methods as well as static methods to test if two messages of the same type are equal:
user.equals(user); // true
user.equals(null); // false
User.equals(user, user); // true
User.equals(user, null); // false User.typeName; // docs.UserAll messages provide methods for serializing and parsing data between the binary and JSON formats. Conformance is ensured by the conformance test suite. Protobuf-ES does not implement the text format.
Here is an example that serializes a message to the binary format, and parses it again:
const user = new User({
firstName: "Homer",
active: true,
});
const bytes: Uint8Array = user.toBinary();
User.fromBinary(bytes);The toBinary and fromBinary methods each take an optional argument with
serialization options.
Serializing to JSON can be done in a similar fashion:
import type { JsonValue } from "@bufbuild/protobuf";
const json: JsonValue = user.toJson();
User.fromJson(json);JsonValue is a type that accurately represents any possible
value in JSON. It can safely be serialized to a string with JSON.stringify.
For convenience, we also provide methods that include the stringify step:
// Same as JSON.stringify(user.toJson());
const json: string = user.toJsonString();
// Same as User.fromJson(JSON.parse(json));
User.fromJsonString(json);The JSON format comes with several options. Note
that if you are using google.protobuf.Any or extensions,
you have to provide a registry with the option typeRegistry.
As a general guide to decide between the binary format and JSON: The JSON format is great for debugging, but the binary format is more resilient to changes. For example, you can rename a field, and still parse binary data serialized with the previous version. In general, the binary format is also more performant than JSON.
To learn about serialization options and other details related to serialization, see the section about advanced serialization.
To check whether a given object is a message, use the function isMessage.
isMessage is mostly equivalent to the instanceof operator. For
example, isMessage(foo, MyMessage) is the same as foo instanceof MyMessage,
and isMessage(foo) is the same as foo instanceof Message.
The advantage of isMessage is that it compares identity by the message type
name, not by class identity. This makes it robust against the dual package
hazard and similar situations, where the same message is duplicated.
To determine if an object is any subtype of Message, pass that object to the
function. To determine if an object is a specific type of Message, pass the
object as well as the type.
import { isMessage } from "@bufbuild/protobuf";
const user = new User({
firstName: "Homer",
});
isMessage(user); // true
isMessage(user, User); // true
isMessage(user, OtherMessageType); // falseFor enumerations, we lean on TypeScript enums. A quick refresher about them:
- It is possible to look up the name for an enum value:
let val: MyEnum = MyEnum.FOO; let name = MyEnum[val]; // => "FOO"
- and to look up an enum value by name:
let val: MyEnum = MyEnum["FOO"];
- TypeScript enums are just plain objects in JavaScript.
- TypeScript enums support aliases - as does protobuf with the
allow_aliasoption.
Note that in Protobuf-ES, all enums are "open", meaning that old generated code can contain a value in an enum field that was added in a new version of the schema. With TypeScript v5 and later, enums are closed in the type system. With earlier versions of TypeScript, they are open.
Extensions can be set on a message using the setExtension function. Provided
we have the following message and extension:
syntax = "proto2";
message User {
extensions 100 to 200;
}
extend User {
optional uint32 age = 100;
}You can set the extension field age like this:
import { setExtension } from "@bufbuild/protobuf";
import { User, age } from "./example_pb.js";
const user = new User();
setExtension(user, age, 77);If the message already has a value for the age extension, the value is replaced.
You can remove an extension from a message with the function clearExtension.
To retrieve an extension value, use getExtension. To check whether an extension
is set, use hasExtension.
import { setExtension, getExtension, hasExtension, clearExtension } from "@bufbuild/protobuf";
setExtension(user, age, 77);
hasExtension(user, age); // true
getExtension(user, age); // 77
clearExtension(user, age);
hasExtension(user, age); // falseNote that getExtension never returns undefined. If the extension is not set,
hasExtension returns false, but getExtension returns the default value,
for example 0 for numeric types, [] for repeated fields, and an empty message
instance for message fields.
Extensions are stored as unknown fields on a message . If you
retrieve an extension value, it is deserialized from the binary unknown field
data. To mutate a value, make sure to store the new value with setExtension
after mutating. For example, let's say we have the extension field
repeated string hobbies = 101, and want to add values:
import { setExtension, getExtension, hasExtension, clearExtension } from "@bufbuild/protobuf";
import { hobbies } from "./example_pb.js";
const h = getExtension(user, hobbies);
h.push("Baking");
h.push("Pottery");
setExtension(user, hobbies, h);If you parse or serialize a message to binary, extensions are automatically included, since they are stored as unknown fields. If you parse or serialize a message to JSON, you have to provide a registry with the extensions you want to include, similar to the well-known type Any.
In the following example, we use the serialization option
typeRegistry to provide extensions:
import { createRegistry } from "@bufbuild/protobuf";
import { age, hobbies } from "./example_pb.js";
const typeRegistry = createRegistry(age, hobbies);
user.toJsonString({ typeRegistry }); // {"[age]":77,"[hobbies]":["Baking","Pottery"]}Extension are commonly used for custom options, which allow to annotate elements in a Protobuf file with arbitrary information.
Custom options are extensions to the google.protobuf.*Options messages defined
in google/protobuf/descriptor.proto.
When a Protobuf compiler parses a file, it converts all elements into descriptors,
and sets custom option values on the option message field of the corresponding
descriptor.
When a plugin is invoked to generate code, it receives the parsed descriptors, and the plugin can read the custom option value using the extension. To see how this works in practice, take a look at the example in our guide for writing plugins.
At this point in time, it is not possible to retrieve custom options from generated code, since Protobuf-ES does not embed the full descriptors in the generated code.
Protocol buffers have a small standard library of well-known types. @bufbuild/protobuf provides all of them as pre-compiled exports.
Expand the list of Well-known types
Note that this list does not include google/protobuf/descriptors.proto, but @bufbuild/protobuf exports all types defined in this file as well.
Some of the well-known types provide additional methods for convenience:
import { Timestamp } from "@bufbuild/protobuf";
// Create an instance from a built-in Date object
let ts = Timestamp.fromDate(new Date(1938, 0, 10));
// Create an instance with the current time
ts = Timestamp.now()
// Convert to a built-in Date object
ts.toDate();import { Any } from "@bufbuild/protobuf";
import { Timestamp } from "@bufbuild/protobuf";
// Pack a message:
let any = Any.pack(user);
any.typeUrl; // type.googleapis.com/docs.User
// Check what an Any contains:
any.is(User); // true
any.is(Timestamp); // false
// Unpack an Any by providing a blank instance:
let user = new User();
any.unpackTo(user); // true
// Alternative: Unpack an Any using a type registry:
const typeRegistry = createRegistry(User, Timestamp);
any.unpack(typeRegistry); // Message of type User
let ts = new Timestamp();
any.unpackTo(ts); // false, you provided an instance of the wrong typeAny stores the message as binary data. To parse or serialize Any to JSON,
you need to provide a registry, similar to extensions.
import { Any, createRegistry, Timestamp } from "@bufbuild/protobuf";
// Pack a Timestamp message in an Any:
const timestamp = Timestamp.now();
const any = Any.pack(timestamp);
// Create a registry so that the Timestamp type can be looked up and converted
// to JSON during serialization:
const typeRegistry = createRegistry(Timestamp);
any.toJsonString({ typeRegistry });google.protobuf.Struct can represent anything JSON can represent. But it is a bit
cumbersome to construct a Struct:
let struct = new Struct({
fields: {
a: {
kind: { case: "numberValue", value: 123 },
},
b: {
kind: { case: "stringValue", value: "abc" },
},
},
});
struct.toJsonString(); // "{a: 123, b: \"abc\"}"We recommend to use fromJson() to construct Struct literals:
let struct = Struct.fromJson({
a: 123,
b: "abc",
});We use the bigint primitive to represent 64-bit integral types, because JavaScript
Number cannot represent the full range of 64-bit numbers.
For the following field definitions:
message User {
string first_name = 1;
string last_name = 2;
bool active = 3;
User manager = 4;
repeated string locations = 5;
map<string, string> projects = 6;
+ uint64 ulong = 4;
+ int64 long = 5;
}You can use bigint as expected:
user.ulong = 123n;
user.long = -123n;
user.ulong + 1n; // 124n
user.long + 1n; // -122nWith the built-in field option jstype = JS_STRING, 64-bit integral fields will
use string instead of bigint:
int64 long = 5 [jstype = JS_STRING]; // will generate `long: number`If bigint is not available in your environment, you can still serialize and
deserialize messages with 64-bit integral fields without losing any data. But
Protobuf-ES will convert those numbers into string. That means you can always
call toString() on a bigint field, and will always receive a string
representation that is suitable to display in a GUI, as a map key, or for similar
purposes.
In case you simply want to set a field value, for example from an HTML form input,
use the provided conversion utility protoInt64:
import { protoInt64 } from "@bufbuild/protobuf";
let input: string | number | bigint = "123";
user.long = protoInt64.parse(input);
user.ulong = protoInt64.uParse(input);If you want to perform arithmetic on bigint fields, you will need to use a
third party library like Long.js.
One of the strong points of Protobuf are its reflection capabilities. In the following sections, we will take a look at the concepts, and how they are implemented in Protobuf-ES.
Descriptors describe Protobuf definitions. Every Protobuf compiler parses source files into descriptors, which are protobuf messages themselves.
For example, the command buf build proto --output set.binpb compiles all Protobuf
files in the directory proto, and writes the message google.protobuf.FileDescriptorSet
to the file set.binpb.
The message google.protobuf.FileDescriptorSet is defined in google/protobuf/descriptor.proto,
along with other messages and ancillary types describing every element of Protobuf
source. Protobuf-ES provides all descriptor messages as exports from @bufbuild/protobuf,
along with all other well-known types.
For a simple example, the following script will read and parse the file created by the compiler command from above, and print the name of each Protobuf file:
import { FileDescriptorSet } from "@bufbuild/protobuf";
import { readFileSync } from "node:fs";
const set = FileDescriptorSet.fromBinary(
readFileSync("./set.binpb"),
);
for (const file of set.file) {
console.log(file.name);
}You can find a deeper dive into the model in Buf's reference about descriptors.
Similar to several other Protobuf implementations, Protobuf-ES provides wrapper
types for the Protobuf descriptor messages that avoid many of their quirks:
The function createDescriptorSet from @bufbuild/protobuf
takes a google.protobuf.FileDescriptorSet as an input, and returns a
DescriptorSet object. This object contains an array
of all files, and map collections for all top-level types in a convenient
wrapped form.
The following table shows how Protobuf descriptor messages map to their wrapped counterparts:
Protobuf message google.protobuf. |
Interface from @bufbuild/protobuf |
|---|---|
FileDescriptorProto |
DescFile |
DescriptorProto |
DescMessage |
FieldDescriptorProto |
DescField, DescExtension |
OneofDescriptorProto |
DescOneof |
EnumDescriptorProto |
DescEnum |
EnumValueDescriptorProto |
DescEnumValue |
ServiceDescriptorProto |
DescService |
MethodDescriptorProto |
DescMethod |
If you write a Protobuf plugin with our framework @bufbuild/protoplugin, you'll see that it provides the wrapped types for the schema to generate. You can find concrete examples in our guide for writing plugins.
Many Protobuf implementations embed descriptors in the generated code so that they are available for reflection. For example, custom options can typically be retrieved from the descriptor at runtime.
Protobuf-ES does not embed full descriptors in the generated code, but a very minimal set of information. The information is sufficient to walk over all fields of a message and access values. Serialization and many other operations in Protobuf-ES are implemented using this information, and you can use it for your own purposes. The entry points are Message types and Enum types.
The following example shows how to iterate over the fields of an arbitrary message:
function walkFields(message: AnyMessage) {
for (const fieldInfo of message.getType().fields.byNumber()) {
const value = message[fieldInfo.localName];
console.log(`field ${fieldInfo.localName}: ${value}`);
}
}
walkFields(user);
// field firstName: Homer
// field lastName: Simpson
// field active: true
// field manager: undefined
// field locations: SPRINGFIELD
// field projects: {"SPP":"Springfield Power Plant"}For a more practical example that covers all cases, you can take a look at the
source of toPlainMessage.
We gave an overview of the message class earlier. Besides the
attributes listed there, message classes actually come with a few more static
properties. The static shape of the generated class is a MessageType,
a representation of the type of a message.
Such a type can actually be created at run time. We can take a peek at the generated code to get some insights:
class User extends Message<User> {
//...
static readonly runtime = proto3;
static readonly typeName = "docs.User";
static readonly fields: FieldList = proto3.util.newFieldList(() => [
{ no: 1, name: "first_name", kind: "scalar", T: 9 /* ScalarType.STRING */ },
{ no: 2, name: "last_name", kind: "scalar", T: 9 /* ScalarType.STRING */ },
{ no: 3, name: "active", kind: "scalar", T: 8 /* ScalarType.BOOL */ },
{ no: 4, name: "manager", kind: "message", T: User },
{ no: 5, name: "locations", kind: "scalar", T: 9 /* ScalarType.STRING */, repeated: true },
{ no: 6, name: "projects", kind: "map", K: 9 /* ScalarType.STRING */, V: {kind: "scalar", T: 9 /* ScalarType.STRING */} },
]);We can observe three properties here:
runtime– will be eitherproto2orproto3, depending on the syntax of the source file.typeName– the fully qualified name of the message, constructed from the package name, a dot, and the original name of the message.fields– all fields of the message are listed here, together with their field number, name and type.
This is actually all the information we need to re-create this message type at run time:
const User = proto3.makeMessageType(
"docs.User",
() => [
{ no: 1, name: "first_name", kind: "scalar", T: 9 /* ScalarType.STRING */ },
{ no: 2, name: "last_name", kind: "scalar", T: 9 /* ScalarType.STRING */ },
{ no: 3, name: "active", kind: "scalar", T: 8 /* ScalarType.BOOL */ },
{ no: 4, name: "manager", kind: "message", T: User },
{ no: 5, name: "locations", kind: "scalar", T: 9 /* ScalarType.STRING */, repeated: true },
{ no: 6, name: "projects", kind: "map", K: 9 /* ScalarType.STRING */, V: {kind: "scalar", T: 9 /* ScalarType.STRING */} },
],
);The resulting User is completely equivalent to the generated TypeScript class. In fact,
this exact piece of code is generated with the plugin option target=js, because if saves us
quite a bit of code size.
Similar to messages, enumerations also include Protobuf metadata. Since TypeScript
enumerations cannot be extended with methods, EnumType is stored
as a symbol property on the enum object. It can be retrieved with the utility
proto3.getEnumType().
EnumType provides the fully qualified name in Protobuf, and
methods to find values by name or integer value.
Here is an example program that prints metadata for a generated enumeration:
import { proto3 } from "@bufbuild/protobuf";
import { MyEnum } from "./generated";
const enumType = proto3.getEnumType(MyEnum);
console.log(enumType.typeName);
for (const value of enumType.values) {
console.log("integer value:", value.no);
console.log("protobuf name:", value.name);
console.log("name in generated code:", value.localName);
}Similar to messages, enumerations can also be created at runtime, via
proto3.makeEnum().
There are some situations where you may want the ability to look up a message
class or other type by type name. For example, google.protobuf.Any stores
an arbitrary message as a type name and binary data.
For this purpose, Protobuf-ES has registries: A simple interface that lets you
find a type by name. You use the convenient function createRegistry to create
one. It takes any of the generated types in the arguments. For example:
import { createRegistry } from "@bufbuild/protobuf";
import { MessageA, MessageB } from "./generated"
const registry = createRegistry(
MessageA,
MessageB,
);
const messageType = registry.findMessage("foo.MessageA");
if (messageType) {
const instance = new messageType();
}Registries are also used during JSON serialization: To convert google.protobuf.Any
and extensions to their JSON representation and back, the
registry is necessary to look up types by name. You can provide a registry in the
serialization option typeRegistry.
Protobuf-ES can create a registry from generated code with createRegistry,
but it can also create a registry from descriptors with the
function createRegistryFromDescriptors.
When a message, enumeration, or extension is retrieved from this registry, the type is created dynamically at runtime. The dynamic types are functionally identical to generated code, and can interact with message data the same way. Dynamic messages are used to interact with message data for types that are not known at compile time.
As an example, let's write a small tool that converts Protobuf message data from the binary format to JSON:
import { createRegistryFromDescriptors } from "@bufbuild/protobuf";
import { readFileSync } from "node:fs";
// The first argument is the path to a `google.protobuf.FileDescriptorSet`,
// the second argument is the fully qualified type name of a message in the set
if (process.argv.length !== 4) {
console.error(`Usage: ${process.argv[1]} path-to-file-descriptor-set message-type-name`);
process.exit(1);
}
const [, , schemaFile, messageTypeName] = process.argv;
// Create the message type at runtime
const registry = createRegistryFromDescriptors(readFileSync(schemaFile));
const messageType = registry.findMessage(messageTypeName);
if (!messageType) {
console.error(`failed to find message type ${messageTypeName}`);
process.exit(1);
}
// Read the binary message data from stdin
const message = messageType.fromBinary(readFileSync(0));
// Write the message as JSON to stdout
console.log(message.toJsonString({
typeRegistry: registry,
prettySpaces: 2,
}));Options for Message.toBinary:
writeUnknownFields?: boolean
By default, unknown fields are included in the serialized output. Setting this option tofalsechanges the behavior to elide unknown fields.writerFactory?: () => IBinaryWriter
A function for specifying a custom implementation to encode binary data.
Options for Message.fromBinary:
readUnknownFields?: boolean
By default, unknown fields are retained during parsing. Setting this option tofalsechanges the behavior to ignore unknown fields.readerFactory?: (bytes: Uint8Array) => IBinaryReader
A function for specifying a custom implementation to decode binary data.
Options for Message.fromJson and Message.fromJsonString:
ignoreUnknownFields?: boolean
By default, unknown properties are rejected. This option overrides this behavior and ignores properties, as well as unrecognized enum string representations.typeRegistry?: IMessageTypeRegistry & Partial<IExtensionRegistry>
A registry to parse extensions and google.protobuf.Any from JSON.
Options for Message.toJson and Message.toJsonString:
emitDefaultValues?: boolean
Fields with default values are omitted by default in JSON output. This option overrides this behavior and outputs fields with their default values.enumAsInteger?: boolean
The name of an enum value is used by default in JSON output. This option overrides the behavior to use the numeric value of the enum value instead.useProtoFieldName?: boolean
Field names are converted to lowerCamelCase by default in JSON output. This option overrides the behavior to use the proto field name instead.typeRegistry?: IMessageTypeRegistry & Partial<IExtensionRegistry>
A registry to convert extensions and google.protobuf.Any to JSON.prettySpaces?: boolean
Only available withtoJsonString. A convenience property for thespaceoption toJSON.stringify, which controls indentation for prettier output. See theJSON.stringifydocs.
Besides toJson and toJsonString, messages also have a toJSON method that is
used by JSON.stringify. See the documentation on MDN
for details on how it works.
We implement this method to ensure that Protobuf messages are always serialized
with the canonical JSON format. Otherwise, JSON.stringify() would
crash on BigInt values, and would not serialize oneof, enumerations, and
other types correctly.
The toJSON method is marked as protected since you should never need to invoke
this function directly.
Serializing a message with JSON.stringify() is equivalent to calling toJsonString
on the message, with the serialization option
emitDefaultValues: true.
When binary message data is parsed, fields that the parser does not recognize are preserved. They are stored on the message as unknown fields, and will be included when the message is serialized again.
This default behavior can be modified with the binary serialization options
readUnknownFields and writeUnknownFields.
Note that extension values are also stored as unknown fields.
Protobuf-ES supports the size-delimited format for messages. It lets you serialize multiple messages to a stream, and parse multiple messages from a stream.
A size-delimited message is a varint size in bytes, followed by exactly that many bytes of a message serialized with the binary format. This implementation is compatible with the counterparts in C++, Java, Go, and others.
The export protoDelimited provides a method to serialize
such a size-delimited message:
import { protoDelimited } from "@bufbuild/protobuf";
const bytes = protoDelimited.enc(new User({ firstName: "John" }));
const user = protoDelimited.dec(User, bytes);To parse size-delimited messages from a stream, the export provides the method
decStream. The method expects an AsyncIterable<Uint8Array> as a stream input,
so it works with Node.js streams out of the box, and can be easily adapted to other
stream APIs:
import { protoDelimited } from "@bufbuild/protobuf";
import { createReadStream, createWriteStream } from "fs";
import { tmpdir } from "os";
import { join } from "path";
// Let's write a couple of messages to a file
const ws = createWriteStream("protoDelimited.bin", {encoding: "binary"});
ws.write(protoDelimited.enc(new User({ firstName: "John" })));
ws.write(protoDelimited.enc(new User({ firstName: "Max" })));
ws.write(protoDelimited.enc(new User({ firstName: "Max" })));
ws.end();
ws.close();
// Now we can parse them from the stream
const readStream = createReadStream("protoDelimited.bin");
for await (const user of protoDelimited.decStream(User, readStream)) {
console.log(user);
}At a low level, the Protobuf binary serialization is implemented with the classes
BinaryReader and BinaryWriter. They
implement the primitives of the Protobuf binary encoding.
Both classes are part of the public API and can be used on their own. The
following example uses BinaryWriter to serialize valid data
for our example message:
import { BinaryWriter } from "@bufbuild/protobuf";
import { User } from "./generated";
const bytes = new BinaryWriter()
// string first_name = 1
.tag(1, WireType.LengthDelimited)
.string("Homer")
// bool active = 3
.tag(3, WireType.Varint)
.bool(true)
.finish();
const user = User.fromBinary(bytes);
user.firstName; // "Homer"
user.active; // trueInternally, the classes use TextEncoder and TextDecoder from the text
encoding API to
encode and decode text as UTF-8. In an environment where this API is unavailable,
your need to bring your own UTF-8 encoder. To do so, you can use the
serialization options writerFactory and
readerFactory to provide your own implementation.
Unfortunately, there is no convenient standard API for base64 encoding in ECMAScript, but it can be very useful when transmitting binary data.
The export protoBase64 provides methods to encode and
decode base64:
import { protoBase64 } from "@bufbuild/protobuf";
import { User } from "./generated";
const user = new User({ firstName: "Joe" });
const bytes: Uint8Array = user.toBinary();
const base64: string = protoBase64.enc(bytes)This type is well suited in case you know the type of a message, but want to allow
an instance to be given in the most flexible way. If you want to offer an API that lets
users provide message data, consider accepting PartialMessage<T>,
so that users can simply give an object literal with only the non-default values
they want. Note that any T is assignable to PartialMessage<T>.
For example, let's say you have a protobuf message User, and you want to provide a
function to your users that processes this message:
export function sendUser(user: PartialMessage<User>) {
// convert partial messages into their full representation if necessary
const u = isMessage(user, User) ? user : new User(user);
// process further...
const bytes = u.toBinary();
}All three examples below are valid input for your function:
sendUser({firstName: "Homer"});
const u = new User();
u.firstName = "Homer";
sendUser(u);
sendUser(new User());PlainMessage<T> represents just the fields of a message, without
their methods.
In contrast to PartialMessage, PlainMessage requires all properties to be
provided. For example:
let plain: PlainMessage<User> = {
firstName: "Homer",
lastName: "Simpson",
active: true,
manager: undefined,
locations: [],
projects: {}
};As such, PlainMessage<T> can be a great fit to use
throughout your business logic, if that business logic is never concerned with
serialization, and does not need instanceof.
Note that any T (assuming T extends Message) is assignable to a variable
of type PlainMessage<T>.
If you want to handle messages of unknown type, the type AnyMessage
provides a convenient index signature to access fields:
const anyMessage: AnyMessage = user;
user["firstName"];Note that any message is assignable to AnyMessage.