Merge branch 'master' into 3.17.x

BUILD

@@ -135,6 +135,7 @@ cc_library(
         "src/google/protobuf/arena.cc",
         "src/google/protobuf/arenastring.cc",
         "src/google/protobuf/extension_set.cc",
+        "src/google/protobuf/field_access_listener.cc",
         "src/google/protobuf/generated_enum_util.cc",
         "src/google/protobuf/generated_message_table_driven_lite.cc",
         "src/google/protobuf/generated_message_util.cc",

CHANGES.txt

@@ -1,3 +1,16 @@
+Unreleased Changes (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
+  C++
+  * Introduce FieldAccessListener.
+  * Stop emitting boilerplate {Copy/Merge}From in each ProtoBuf class
+  * Fixed some uninitialized variable warnings in generated_message_reflection.cc.
+
+  Kotlin:
+  * Add extension functions related to ByteString to c.g.protobuf.kotlin.
+
+  Java
+  * Fixed parser to check that we are at a proper limit when a sub-message has
+    finished parsing.
+
 2021-05-25 version 3.17.2 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
   Kotlin
   * Fix duplicate class error (#8653)
@@ -17,6 +30,7 @@
   * Fixed JSON parser to allow multiple values from the same oneof as long as
     all but one are null.
 
+
 2021-05-07 version 3.17.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
 
   Protocol Compiler
@@ -63,13 +77,26 @@
   * Added a conformance test for the case of multiple fields from the same
     oneof.
 
+2021-04-06 version 3.16.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
   Other
   * Opensourcing kotlin protos (#8272)
   * Use a newer version of rules_proto, with the new rule `proto_descriptor_set` (#8469)
 
-3.16.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
-
   C++
+  * Fix compiler warnings issue found in conformance_test_runner #8189 (#8190)
+  * Fix MinGW-w64 build issues. (#8286)
+  * [Protoc] C++ Resolved an issue where NO_DESTROY and CONSTINIT are in incorrect order (#8296)
+  * Fix PROTOBUF_CONSTINIT macro redefinition (#8323)
+  * Delete StringPiecePod (#8353)
+  * Fix gcc error: comparison of unsigned expression in '>= 0' is always … (#8309)
+  * Fix cmake install on iOS (#8301)
+  * Create a CMake option to control whether or not RTTI is enabled (#8347)
+  * Fix endian.h location on FreeBSD (#8351)
+  * Refactor util::Status (#8354)
+  * Make util::Status more similar to absl::Status (#8405)
+  * Fix -Wsuggest-destructor-override for generated C++ proto classes. (#8408)
+  * Refactor StatusOr and StringPiece (#8406)
+  * Refactor uint128 (#8416)
   * The ::pb namespace is no longer exposed due to conflicts.
   * Allow MessageDifferencer::TreatAsSet() (and friends) to override previous
     calls instead of crashing.
@@ -94,20 +121,49 @@
   * util::DefaultFieldComparator will be final in a future version of protobuf.
     Subclasses should inherit from SimpleFieldComparator instead.
 
-  Java:
+  C#
+  * Add .NET 5 target and improve WriteString performance with SIMD (#8147)
+
+  Java
+  * deps: update JUnit and Truth (#8319)
   * Detect invalid overflow of byteLimit and return InvalidProtocolBufferException as documented.
   * Exceptions thrown while reading from an InputStream in parseFrom are now
     included as causes.
   * Support potentially more efficient proto parsing from RopeByteStrings.
   * Clarify runtime of ByteString.Output.toStringBuffer().
+  * Added UnsafeByteOperations to protobuf-lite (#8426)
+
+  JavaScript
+  * Make Any.pack() chainable.
 
   Python
+  * Fix some constness / char literal issues being found by MSVC standard conforming mode (#8344)
+  * Switch on "new" buffer API (#8339)
+  * Enable crosscompiling aarch64 python wheels under dockcross manylinux docker image (#8280)
   * Fixed a bug in text format where a trailing colon was printed for repeated field.
   * When TextFormat encounters a duplicate message map key, replace the current
     one instead of merging.
 
-  JavaScript
-  * Make Any.pack() chainable.
+  Objective-C
+  * Move the class map to a CFDictionary. (#8328)
+
+  PHP
+  * read_property() handler is not supposed to return NULL (#8362)
+  * Changed parameter type from long to integer (#7613)
+  * fix: README supported PHP version for C extension (#8236)
+
+  Ruby
+  * Fixed quadratic memory usage when appending to arrays. (#8364)
+  * Fixed memory leak of Ruby arena objects. (#8461)
+  * Add support for proto3 json_name in compiler and field definitions. (#8356)
+
+  Other
+  * Some doc on AOT compilation and protobuf (#8294)
+  * [CMake] Ability to pass options to protoc executable from cmake (#8374)
+  * Add --fatal_warnings flag to treat warnings as errors (#8131)
+  * [bazel] Remove deprecated way to depend on googletest (#8396)
+  * add error returns missing from protoc to prevent it from exiting with… (#8409)
+
 
 2021-04-07 version 3.15.8 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
 

Makefile.am

@@ -1427,7 +1427,8 @@ EXTRA_DIST = $(@DIST_LANG@_EXTRA_DIST)   \
   third_party/zlib.BUILD                 \
   third_party/wyhash/LICENSE             \
   third_party/wyhash/wyhash.h            \
-  util/python/BUILD
+  util/python/BUILD                      \
+  internal.bzl
 
 
 # Deletes all the files generated by autogen.sh.

README.md

@@ -52,18 +52,18 @@ Protobuf supports several different programming languages. For each programming
 language, you can find instructions in the corresponding source directory about
 how to install protobuf runtime for that specific language:
 
-| Language                             | Source                                                      | Ubuntu | MacOS | Windows |
-|--------------------------------------|-------------------------------------------------------------|--------|-------|---------|
-| C++ (include C++ runtime and protoc) | [src](src)                                                  | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-cpp_distcheck.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fcpp_distcheck%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-bazel.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fbazel%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-dist_install.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fdist_install%2Fcontinuous) | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-cpp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fcpp%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-cpp_distcheck.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fcpp_distcheck%2Fcontinuous) | [![Build status](https://ci.appveyor.com/api/projects/status/73ctee6ua4w2ruin?svg=true)](https://ci.appveyor.com/project/protobuf/protobuf) |
-| Java                                 | [java](java)                                                | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-java_compatibility.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fjava_compatibility%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-java_jdk7.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fjava_jdk7%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-java_oracle7.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fjava_oracle7%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-java_linkage_monitor.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fjava_linkage_monitor%2Fcontinuous) | | |
-| Python                               | [python](python)                                            | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python27.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython27%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python35.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython35%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python36.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython36%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python37.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython37%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python_compatibility.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython_compatibility%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python27_cpp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython27_cpp%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python35_cpp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython35_cpp%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python36_cpp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython36_cpp%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python37_cpp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython37_cpp%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-python-release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fpython_release%2Fcontinuous) | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-python.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fpython%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-python_cpp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fpython_cpp%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-python-release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fpython_release%2Fcontinuous) | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/windows-python-release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fwindows%2Fpython_release%2Fcontinuous) |
-| Objective-C                          | [objectivec](objectivec)                                    | | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-objectivec_cocoapods_integration.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fobjectivec_cocoapods_integration%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-objectivec_ios_debug.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fobjectivec_ios_debug%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-objectivec_ios_release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fobjectivec_ios_release%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-objectivec_osx.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fobjectivec_osx%2Fcontinuous) | |
-| C#                                   | [csharp](csharp)                                            | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-csharp.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fcsharp%2Fcontinuous) | | [![Build status](https://ci.appveyor.com/api/projects/status/73ctee6ua4w2ruin?svg=true)](https://ci.appveyor.com/project/protobuf/protobuf)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/windows-csharp-release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fwindows%2Fcsharp_release%2Fcontinuous) |
-| JavaScript                           | [js](js)                                                    | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-javascript.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fjavascript%2Fcontinuous) | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-javascript.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fjavascript%2Fcontinuous) | |
-| Ruby                                 | [ruby](ruby)                                                | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-ruby23.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fruby23%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-ruby24.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fruby24%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-ruby25.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fruby25%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-ruby26.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fruby26%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-ruby-release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fruby_release%2Fcontinuous) | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-ruby23.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fruby23%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-ruby24.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fruby24%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-ruby25.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fruby25%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-ruby26.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fruby26%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-ruby-release.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fruby_release%2Fcontinuous) | |
-| Go                                   | [protocolbuffers/protobuf-go](https://github.com/protocolbuffers/protobuf-go)       | | | |
-| PHP                                  | [php](php)                                                  | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-php_all.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fphp_all%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-32-bit.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2F32-bit%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/linux-php80.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fubuntu%2Fphp80%2Fcontinuous) | [![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-php5.6_mac.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fphp5.6_mac%2Fcontinuous)<br/>[![Build status](https://storage.googleapis.com/protobuf-kokoro-badges/status-badge/macos-php7.0_mac.png)](https://fusion.corp.google.com/projectanalysis/current/KOKORO/prod:protobuf%2Fgithub%2Fmaster%2Fmacos%2Fphp7.0_mac%2Fcontinuous) | |
-| Dart                                 | [dart-lang/protobuf](https://github.com/dart-lang/protobuf) | [![Build Status](https://travis-ci.org/dart-lang/protobuf.svg?branch=master)](https://travis-ci.org/dart-lang/protobuf) | | |
+| Language                             | Source                                                      |
+|--------------------------------------|-------------------------------------------------------------|
+| C++ (include C++ runtime and protoc) | [src](src)                                                  |
+| Java                                 | [java](java)                                                |
+| Python                               | [python](python)                                            |
+| Objective-C                          | [objectivec](objectivec)                                    |
+| C#                                   | [csharp](csharp)                                            |
+| JavaScript                           | [js](js)                                                    |
+| Ruby                                 | [ruby](ruby)                                                |
+| Go                                   | [protocolbuffers/protobuf-go](https://github.com/protocolbuffers/protobuf-go)|
+| PHP                                  | [php](php)                                                  |
+| Dart                                 | [dart-lang/protobuf](https://github.com/dart-lang/protobuf) |
 
 Quick Start
 -----------

autogen.sh

@@ -30,8 +30,12 @@ set -ex
 
 # The absence of a m4 directory in googletest causes autoreconf to fail when
 # building under the CentOS docker image. It's a warning in regular build on
-# Ubuntu/gLinux as well.
-mkdir -p third_party/googletest/m4
+# Ubuntu/gLinux as well. (This is only needed if git submodules have been
+# initialized, which is typically only needed for testing; see the installation
+# instructions for details.)
+if test -d third_party/googletest; then
+  mkdir -p third_party/googletest/m4
+fi
 
 # TODO(kenton):  Remove the ",no-obsolete" part and fix the resulting warnings.
 autoreconf -f -i -Wall,no-obsolete