Protocol Buffers v3.10.0

C++

• Switch the proto parser to the faster MOMI parser.
• Properly escape Struct keys in the proto3 JSON serializer.
• Fix crash on uninitialized map entries.
• Informed the compiler of has-bit invariant to produce better code
• Unused imports of files defining descriptor extensions will now be reported
• Add proto2::util::RemoveSubranges to remove multiple subranges in linear time.
• Added BaseTextGenerator::GetCurrentIndentationSize()
• Made implicit weak fields compatible with the Apple linker
• Support 32 bit values for ProtoStreamObjectWriter to Struct.
• Removed the internal-only header coded_stream_inl.h and the internal-only methods defined there.
• Enforced no SWIG wrapping of descriptor_database.h (other headers already had this restriction).
• Implementation of the equivalent of the MOMI parser for serialization. This removes one of the two serialization routines, by making the fast array serialization routine completely general. SerializeToCodedStream can now be implemented in terms of the much much faster array serialization. The array serialization regresses slightly, but when array serialization is not possible this wins big.
• Do not convert unknown field name to snake case to accurately report error.
• Fix a UBSAN warnings. (#6333)
• Add podspec for C++ (#6404)
• protoc: fix source code info location for missing label (#6436)
• C++ Add move constructor for Reflection's SetString (#6477)

Java

This release has a known issue on Android API level <26 (#6718) if you are using protobuf-java in Android. protobuf-javalite users will be fine.

• Call loadDescriptor outside of synchronized block to remove one possible source of deadlock.
• Have oneof enums implement a separate interface (other than EnumLite) for clarity.
• Opensource Android Memory Accessors
• Update TextFormat to make use of the new TypeRegistry.
• Support getFieldBuilder and getRepeatedFieldBuilder in ExtendableBuilder
• Update JsonFormat to make use of the new TypeRegistry.
• Add proguard config generator for GmmBenchmarkSuiteLite.
• Change ProtobufArrayList to use Object[] instead of ArrayList for 5-10% faster parsing
• Implement ProtobufArrayList.add(E) for 20% (5%-40%) faster overall protolite2 parsing
• Make a copy of JsonFormat.TypeRegistry at the protobuf top level package. This will eventually replace JsonFormat.TypeRegistry.
• Fix javadoc warnings in generated files (#6231)
• Java: Add Automatic-Module-Name entries to the Manifest (#6568)

Python

• Add descriptor methods in descriptor_pool are deprecated.
• Uses explicit imports to prevent multithread test failures in py3.
• Added delitem for Python extension dict
• Update six version to 1.12.0 and fix legacy_create_init issue (#6391)

JavaScript

• Remove deprecated boolean option to getResultBase64String().
• Fix sint64 zig-zag encoding.
• Simplify hash64 string conversion to avoid DIGIT array. Should reduce overhead if these functions aren't used, and be more efficient by avoiding linear array searches.
• Change the parameter types of binaryReaderFn in ExtensionFieldBinaryInfo to (number, ?, ?).
• Create dates.ts and time_of_days.ts to mirror Java versions. This is a near-identical conversion of c.g.type.util.{Dates,TimeOfDays} respectively.
• Migrate moneys to TypeScript.

PHP

• Fix incorrect leap day for Timestamp (#6696)
• Initialize well known type values (#6713)

Ruby

Update 2019-10-04: Ruby release has been rolled back due to grpc/grpc#20471. We will upload a new version once the bug is fixed.

• Fix scope resolution for Google namespace (#5878)
• Support hashes for struct initializers (#5716)
• Optimized away the creation of empty string objects. (#6502)
• Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
• Optimized layout_mark() for Ruby (#6521)
• Optimization for layout_init() (#6547)
• Fix for GC of Ruby map frames. (#6533)
• Fixed leap year handling by reworking upb_mktime() -> upb_timegm(). (#6695)

Objective C

• Remove OSReadLittle* due to alignment requirements (#6678)
• Don't use unions and instead use memcpy for the type swaps. (#6672)

Other

• Override CocoaPods module to lowercase (#6464)

Protocol Buffers v3.9.2

Objective-C

• Remove OSReadLittle* due to alignment requirements. (#6678)
• Don't use unions and instead use memcpy for the type swaps. (#6672)

Protocol Buffers v3.10.0-rc1

C++

• Switch the proto parser to the faster MOMI parser.
• Properly escape Struct keys in the proto3 JSON serializer.
• Fix crash on uninitialized map entries.
• Informed the compiler of has-bit invariant to produce better code
• Unused imports of files defining descriptor extensions will now be reported
• Add proto2::util::RemoveSubranges to remove multiple subranges in linear time.
• Added BaseTextGenerator::GetCurrentIndentationSize()
• Made implicit weak fields compatible with the Apple linker
• Support 32 bit values for ProtoStreamObjectWriter to Struct.
• Removed the internal-only header coded_stream_inl.h and the internal-only methods defined there.
• Enforced no SWIG wrapping of descriptor_database.h (other headers already had this restriction).
• Implementation of the equivalent of the MOMI parser for serialization. This removes one of the two serialization routines, by making the fast array serialization routine completely general. SerializeToCodedStream can now be implemented in terms of the much much faster array serialization. The array serialization regresses slightly, but when array serialization is not possible this wins big.
• Do not convert unknown field name to snake case to accurately report error.
• Fix a UBSAN warnings. (#6333)
• Add podspec for C++ (#6404)
• protoc: fix source code info location for missing label (#6436)
• C++ Add move constructor for Reflection's SetString (#6477)

Java

• Call loadDescriptor outside of synchronized block to remove one possible source of deadlock.
• Have oneof enums implement a separate interface (other than EnumLite) for clarity.
• Opensource Android Memory Accessors
• Update TextFormat to make use of the new TypeRegistry.
• Support getFieldBuilder and getRepeatedFieldBuilder in ExtendableBuilder
• Update JsonFormat to make use of the new TypeRegistry.
• Add proguard config generator for GmmBenchmarkSuiteLite.
• Change ProtobufArrayList to use Object[] instead of ArrayList for 5-10% faster parsing
• Implement ProtobufArrayList.add(E) for 20% (5%-40%) faster overall protolite2 parsing
• Make a copy of JsonFormat.TypeRegistry at the protobuf top level package. This will eventually replace JsonFormat.TypeRegistry.
• Fix javadoc warnings in generated files (#6231)
• Java: Add Automatic-Module-Name entries to the Manifest (#6568)

Python

• Add descriptor methods in descriptor_pool are deprecated.
• Uses explicit imports to prevent multithread test failures in py3.
• Added delitem for Python extension dict
• Update six version to 1.12.0 and fix legacy_create_init issue (#6391)

JavaScript

• Remove deprecated boolean option to getResultBase64String().
• Fix sint64 zig-zag encoding.
• Simplify hash64 string conversion to avoid DIGIT array. Should reduce overhead if these functions aren't used, and be more efficient by avoiding linear array searches.
• Change the parameter types of binaryReaderFn in ExtensionFieldBinaryInfo to (number, ?, ?).
• Create dates.ts and time_of_days.ts to mirror Java versions. This is a near-identical conversion of c.g.type.util.{Dates,TimeOfDays} respectively.
• Migrate moneys to TypeScript.

Ruby

• Fix scope resolution for Google namespace (#5878)
• Support hashes for struct initializers (#5716)
• Optimized away the creation of empty string objects. (#6502)
• Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
• Optimized layout_mark() for Ruby (#6521)
• Optimization for layout_init() (#6547)
• Fix for GC of Ruby map frames. (#6533)

Other

• Override CocoaPods module to lowercase (#6464)

Protocol Buffers v3.9.1

Python

• Drop building wheel for python 3.4 (#6406)

Csharp

• Fix binary compatibility in 3.9.0 (delisted) FieldCodec factory methods (#6380)

Protocol Buffers v3.9.0

C++

• Optimize and simplify implementation of RepeatedPtrFieldBase
• Don't create unnecessary unknown field sets.
• Remove branch from accessors to repeated field element array.
• Added delimited parse and serialize util.
• Reduce size by not emitting constants for fieldnumbers
• Fix a bug when comparing finite and infinite field values with explicit tolerances.
• TextFormat::Parser should use a custom Finder to look up extensions by number if one is provided.
• Add MessageLite::Utf8DebugString() to make MessageLite more compatible with Message.
• Fail fast for better performance in DescriptorPool::FindExtensionByNumber() if descriptor has no defined extensions.
• Adding the file name to help debug colliding extensions
• Added FieldDescriptor::PrintableNameForExtension() and DescriptorPool::FindExtensionByPrintableName().
  The latter will replace Reflection::FindKnownExtensionByName().
• Replace NULL with nullptr
• Created a new Add method in repeated field that allows adding a range of elements all at once.
• Enabled enum name-to-value mapping functions for C++ lite
• Avoid dynamic initialization in descriptor.proto generated code
• Move stream functions to MessageLite from Message.
• Move all zero_copy_stream functionality to io_lite.
• Do not create array of matched fields for simple repeated fields
• Enabling silent mode by default to reduce make compilation noise. (#6237)

Java

• Expose TextFormat.Printer and make it configurable. Deprecate the static methods.
• Library for constructing google.protobuf.Struct and google.protobuf.Value
• Make OneofDescriptor extend GenericDescriptor.
• Expose streamingness of service methods from MethodDescriptor.
• Fix a bug where TextFormat fails to parse Any filed with > 1 embedded message sub-fields.
• Establish consistent JsonFormat behavior for nulls in oneofs, regardless of order.
• Update GSON version to 3.8.5. (#6268)
• Add protobuf_java_lite Bazel target. (#6177)

Python

• Change implementation of Name() for enums that allow aliases in proto2 in Python
  to be in line with claims in C++ implementation (to return first value).
• Explicitly say what field cannot be set when the new value fails a type check.
• Duplicate register in descriptor pool will raise errors
• Add slots to all well_known_types classes, custom attributes are not allowed anymore.
• text_format only present 8 valid digits for float fields by default

JavaScript

• Add Oneof enum to the list of goog.provide

PHP

• Rename get/setXXXValue to get/setXXXWrapper. (#6295)

Ruby

• Remove to_hash methods. (#6166)

Protocol Buffers v3.9.0-rc1

Merge pull request #6298 from TeBoring/3.9.x-merge

Merge master into 3.9.x

Protocol Buffers v3.8.0

C++

• Use std::atomic in case of myriad2 platform
• Always declare enums to be int-sized
• Added DebugString() and ShortDebugString() methods on MessageLite
• Specialized different parse loop control flows
• Make hasbits potentially in register. The or's start forming an obstacle because it's a read modify store on the same mem address on each iteration.
• Move to an internal MACRO for parser validity checks.
• Improve map parsing performance.
• Make MergePartialFromCodedStream non virtual. This allows direct calls, potential inlining and is also a code health improvement
• Add an overall limit to parse_context to prevent reading past it. This allows to remove a annoying level of indirection.
• Fix a mistake, we shouldn't verify map key/value strings for utf8 in opt mode for proto2.
• Further improvements to cut binary size.
• Prepare to make MergePartialFromCodedStream non-virtual.
• A report on some interesting behavior change in python (caused by b/27494216) made me realize there is a check that needs to be done in case the parse ended on a end group tag.
• Add a note of caution to the comments around skip in CodedOutputStream.
• Simplify end check.
• Add overload for ParseMessage for MessageLite/Message types. If the explicit type is not known inlining won't help de-virtualizing the virtual call.
• Reduce linker input. It turns out that ParseMessage is not inlined, producing template instantiations that are used only once and save nothing but cost more.
• Improve the parser.
• [c++17] Changed proto2::RepeatedPtrField iterators to no longer derive from the deprecated std::iterator class.
• Change the default value of case_insensitive_enum_parsing to false for JsonStringToMessage.
• Add a warning if a field name doesn't match the style guide.
• Fix TextFormat not round-trip correctly when float value is max float.
• Added locationed info for some errors at compiler
• Python reserved keywords are now working with getattr()/setattr() for most descriptors.
• Added AllowUnknownField() in text_format
• Append '_' to C++ reserved keywords for message, enum, extension
• Fix MSVC warning C4244 in protobuf's parse_context.h.
• Updating Iterators to be compatible with C++17 in MSVC.
• Use capability annotation in mutex.h
• Fix "UndefinedBehaviorSanitizer: cfi-bad-type"
• CriticalSectionLock class as a lightweight replacement for std::mutex on Windows platforms.
• Removed vestigial wire_format_lite_inl.h

C#

• Added System.Memory dependency.

Java

• Make Java protoc code generator ignore optimize_for LITE_RUNTIME. Users should instead use the Java lite protoc plugin.
• Change Extension getMessageDefaultInstance() to return Message instead of MessageLite.
• Prevent malicious input streams from leaking buffers for ByteString or ByteBuffer parsing.
• Release new Javalite runtime.
• Show warning in case potential file name conflict.
• Allow Java reserved keywords to be used in extensions.
• Added setAllowUnknownFields() in text format
• Add memoization to ExtensionRegistryLite.getEmptyRegistry()
• Improve performance of CodedOutputStream.writeUInt32NoTag
• Add an optimized mismatch-finding algorithm to UnsafeUtil.
• When serializing uint32 varints, check that we have MAX_VARINT32_SIZE bytes left, not just MAX_VARINT_SIZE.
• Minor optimization to RopeByteString.PieceIterator

JavaScript

• Simplify generated toObject code when the default value is used.

Python

• Changes implementation of Name() for enums that allow aliases in proto2 in Python to be in line with claims in C++ implementation (to return first value).
• Added double_format option in text format printer.
• Added iter and contains to extension dict
• Added allow_unknown_field option in python text format parser
• Fixed Timestamp.ToDatetime() loses precision issue
• Support unknown field in text format printer.
• Float field will be convert to inf if bigger than struct.unpack('f', b'\xff\xff\x7f\x7f')[0] which is about 3.4028234664e+38,
  convert to -inf if smaller than -3.4028234664e+38
• Allowed casting str->bytes in Message.setstate

Ruby

• Helper methods to get enum name for Ruby.

Protocol Buffers v3.8.0-rc1

C++

• Use std::atomic in case of myriad2 platform
• Always declare enums to be int-sized
• Added DebugString() and ShortDebugString() methods on MessageLite
• Specialized different parse loop control flows
• Make hasbits potentially in register. The or's start forming an obstacle because it's a read modify store on the same mem address on each iteration.
• Move to an internal MACRO for parser validity checks.
• Improve map parsing performance.
• Make MergePartialFromCodedStream non virtual. This allows direct calls, potential inlining and is also a code health improvement
• Add an overall limit to parse_context to prevent reading past it. This allows to remove a annoying level of indirection.
• Fix a mistake, we shouldn't verify map key/value strings for utf8 in opt mode for proto2.
• Further improvements to cut binary size.
• Prepare to make MergePartialFromCodedStream non-virtual.
• A report on some interesting behavior change in python (caused by b/27494216) made me realize there is a check that needs to be done in case the parse ended on a end group tag.
• Add a note of caution to the comments around skip in CodedOutputStream.
• Simplify end check.
• Add overload for ParseMessage for MessageLite/Message types. If the explicit type is not known inlining won't help de-virtualizing the virtual call.
• Reduce linker input. It turns out that ParseMessage is not inlined, producing template instantiations that are used only once and save nothing but cost more.
• Improve the parser.
• [c++17] Changed proto2::RepeatedPtrField iterators to no longer derive from the deprecated std::iterator class.
• Change the default value of case_insensitive_enum_parsing to false for JsonStringToMessage.
• Add a warning if a field name doesn't match the style guide.
• Fix TextFormat not round-trip correctly when float value is max float.
• Added locationed info for some errors at compiler
• Python reserved keywords are now working with getattr()/setattr() for most descriptors.
• Added AllowUnknownField() in text_format
• Append '_' to C++ reserved keywords for message, enum, extension
• Fix MSVC warning C4244 in protobuf's parse_context.h.
• Updating Iterators to be compatible with C++17 in MSVC.
• Use capability annotation in mutex.h
• Fix "UndefinedBehaviorSanitizer: cfi-bad-type"
• CriticalSectionLock class as a lightweight replacement for std::mutex on Windows platforms.
• Removed vestigial wire_format_lite_inl.h

C#

• Added System.Memory dependency.

Java

• Make Java protoc code generator ignore optimize_for LITE_RUNTIME. Users should instead use the Java lite protoc plugin.
• Change Extension getMessageDefaultInstance() to return Message instead of MessageLite.
• Prevent malicious input streams from leaking buffers for ByteString or ByteBuffer parsing.
• Release new Javalite runtime.
• Show warning in case potential file name conflict.
• Allow Java reserved keywords to be used in extensions.
• Added setAllowUnknownFields() in text format
• Add memoization to ExtensionRegistryLite.getEmptyRegistry()
• Improve performance of CodedOutputStream.writeUInt32NoTag
• Add an optimized mismatch-finding algorithm to UnsafeUtil.
• When serializing uint32 varints, check that we have MAX_VARINT32_SIZE bytes left, not just MAX_VARINT_SIZE.
• Minor optimization to RopeByteString.PieceIterator

JavaScript

• Simplify generated toObject code when the default value is used.

Python

• Changes implementation of Name() for enums that allow aliases in proto2 in Python to be in line with claims in C++ implementation (to return first value).
• Added double_format option in text format printer.
• Added iter and contains to extension dict
• Added allow_unknown_field option in python text format parser
• Fixed Timestamp.ToDatetime() loses precision issue
• Support unknown field in text format printer.
• Float field will be convert to inf if bigger than struct.unpack('f', b'\xff\xff\x7f\x7f')[0] which is about 3.4028234664e+38,
  convert to -inf if smaller than -3.4028234664e+38
• Allowed casting str->bytes in Message.setstate

Ruby

• Helper methods to get enum name for Ruby.

Protocol Buffers v3.7.1

C++

• Avoid linking against libatomic in prebuilt protoc binaries (#5875)
• Avoid marking generated C++ messages as final, though we will do this in a future release (#5928)
• Miscellaneous build fixes

JavaScript

• Fixed redefinition of global variable f (#5932)

Ruby

• Replace strptime with custom implementation (#5906)
• Fixed the bug that bytes fields cannot be larger than 16000 bytes (#5924)
• Miscellaneous bug fixes

PHP

• Replace strptime with custom implementation (#5906)
• Fixed the bug that bytes fields cannot be larger than 16000 bytes (#5924)

Protocol Buffers v3.7.0

C++

• Introduced new MOMI (maybe-outside-memory-interval) parser.
• Add an option to json_util to parse enum as case-insensitive. In the future, enum parsing in json_util will become case-sensitive.
• Added conformance test for enum aliases
• Added support for --cpp_out=speed:...
• Added use of C++ override keyword where appropriate
• Many other cleanups and fixes.

Java

• Fix illegal reflective access warning in JDK 9+
• Add BOM

Python

• Added Python 3.7 compatibility.
• Modified ParseFromString to return bytes parsed .
• Introduce Proto C API.
• FindFileContainingSymbol in descriptor pool is now able to find field and enum values.
• reflection.MakeClass() and reflection.ParseMessage() are deprecated.
• Added DescriptorPool.FindMethodByName() method in pure python (c extension alreay has it)
• Flipped proto3 to preserve unknown fields by default.
• Added support for memoryview in python3 proto message parsing.
• Added MergeFrom for repeated scalar fields in c extension (pure python already has it)
• Surrogates are now rejected at setters in python3.
• Added public unknown field API.
• RecursionLimit is also set to max if allow_oversize_protos is enabled.
• Disallow duplicate scalars in proto3 text_format parse.
• Fix some segment faults for c extension map field.

PHP

• Most issues for json encoding/decoding in the c extension have been fixed. There are still some edge cases not fixed. For more details, check conformance/failure_list_php_c.txt.
• Supports php 7.3
• Added helper methods to convert between enum values and names.
• Allow setting/getting wrapper message fields using primitive values.
• Various bug fixes.

Ruby

• Ruby 2.6 support.
• Drops support for ruby < 2.3.
• Most issues for json encoding/decoding in the c extension have been fixed. There are still some edge cases not fixed. For more details, check conformance/failure_list_ruby.txt.
• Json parsing can specify an option to ignore unknown fields: msg.decode_json(data, {ignore_unknown_fields: true}).
• Added support for proto2 syntax (partially).
• Various bug fixes.

C#

• More support for FieldMask include merge, intersect and more.
• Increasing the default recursion limit to 100.
• Support loading FileDescriptors dynamically.
• Provide access to comments from descriptors.
• Added Any.Is method.
• Compatible with C# 6
• Added IComparable and comparison operators on Timestamp.

Objective-C

• Add ability to introspect list of enum values (#4678)
• Copy the value when setting message/data fields (#5215)
• Support suppressing the objc package prefix checks on a list of files (#5309)
• More complete keyword and NSObject method (via categories) checks for field names, can result in more fields being rename, but avoids the collisions at runtime (#5289)
• Small fixes to TextFormat generation for extensions (#5362)
• Provide more details/context in deprecation messages (#5412)
• Array/Dictionary enumeration blocks NS_NOESCAPE annotation for Swift (#5421)
• Properly annotate extensions for ARC when their names imply behaviors (#5427)
• Enum alias name collision improvements (#5480)