release-1.11.3

The Apache Avro community is pleased to announce the release of Avro 1.11.3!

This release addresses 39 Avro JIRA.

All signed release artifacts, signatures and verification instructions can
be found here: https://avro.apache.org/releases.html

This is a minor release, specifically addressing known issues with the 1.11.2 release, but also contains version bumps and doc fixes[1].

Language SDK / Convenience artifacts

In addition, language-specific release artifacts are available:

• C#: https://www.nuget.org/packages/Apache.Avro/1.11.3
• Java: https://repo1.maven.org/maven2/org/apache/avro/avro/1.11.3/
• Javascript: https://www.npmjs.com/package/avro-js/v/1.11.3
• Perl: https://metacpan.org/release/Avro
• Python 3: https://pypi.org/project/avro/1.11.3
• Ruby: https://rubygems.org/gems/avro/versions/1.11.3
• Rust: https://crates.io/crates/apache-avro/0.16.0

Thanks to everyone for contributing!

Apache Avro 1.11.2

The Apache Avro community is pleased to announce the release of Avro 1.11.2!

This release addresses 89 Avro JIRA.

Highlights

C#

• AVRO-3434: Support logical schemas in reflect reader and writer
• AVRO-3670: Add NET 7.0 support
• AVRO-3724: Fix C# JsonEncoder for nested array of records
• AVRO-3756: Add a method to return types instead of writing them to disk

C++

• AVRO-3601: C++ API header contains breaking include
• AVRO-3705: C++17 support

Java

• AVRO-2943: Add new GenericData String/Utf8 ARRAY comparison test
• AVRO-2943: improve GenericRecord MAP type comparison
• AVRO-3473: Use ServiceLoader to discover Conversion
• AVRO-3536: Inherit conversions for Union type
• AVRO-3597: Allow custom readers to override string creation
• AVRO-3560: Throw SchemaParseException on dangling content beyond end of schema
• AVRO-3602: Support Map(with non-String keys) and Set in ReflectDatumReader
• AVRO-3676: Produce valid toString() for UUID JSON
• AVRO-3698: SpecificData.getClassName must replace reserved words
• AVRO-3700: Publish Java SBOM artifacts with CycloneDX
• AVRO-3783: Read LONG length for bytes, only allow INT sizes
• AVRO-3706: accept space in folder name

Python

• AVRO-3761: Fix broken validation of nullable UUID field
• AVRO-3229: Raise on invalid enum default only if validation enabled
• AVRO-3622: Fix compatibility check for schemas having or missing namespace
• AVRO-3669: Add py.typed marker file (PEP561 compliance)
• AVRO-3672: Add CI testing for Python 3.11
• AVRO-3680: allow to disable name validation

Ruby

• AVRO-3775: Fix decoded default value of logical type
• AVRO-3697: Test against Ruby 3.2
• AVRO-3722: Eagerly initialize instance variables for better inline cache hits

Rust

• Many, many bug fixes and implementation progress in this experimental SDK.
• Rust CI builds and lints are passing, and has been released to crates.io as version 0.15.0

In addition:

• Upgrade dependencies to latest versions, including CVE fixes.
• Testing and build improvements.
• Performance fixes, other bug fixes, better documentation and more...

Language SDK / Convenience artifacts

• C#: https://www.nuget.org/packages/Apache.Avro/1.11.2
• Java: https://repo1.maven.org/maven2/org/apache/avro/avro/1.11.2/
• Javascript: https://www.npmjs.com/package/avro-js/v/1.11.2
• Perl: https://metacpan.org/release/Avro
• Python 3: https://pypi.org/project/avro/1.11.2
• Ruby: https://rubygems.org/gems/avro/versions/1.11.2
• Rust: https://crates.io/crates/apache-avro/0.15.0

Known issues

• AVRO-3789 [Java]: Problem when comparing empty MAP types.

Thanks to everyone for contributing!

Apache Avro 1.11.1

The Apache Avro community is pleased to announce the release of Avro 1.11.1!

Most interesting

This release includes 256 Jira issues, including some interesting features:

Avro specification

• AVRO-3436 Clarify which names are allowed to be qualified with namespaces
• AVRO-3370 Inconsistent behaviour on types as invalid names
• AVRO-3275 Clarify how fullnames are created, with example
• AVRO-3257 IDL: add syntax to create optional fields
• AVRO-2019 Improve docs for logical type annotation

C++

• AVRO-2722 Use of boost::mt19937 is not thread safe

C#

• AVRO-3383 Many completed subtasks for modernizing C# coding style
• AVRO-3481 Input and output variable type mismatch
• AVRO-3475 Enforce time-millis and time-micros specification
• AVRO-3469 Build and test using .NET SDK 7.0
• AVRO-3468 Default values for logical types not supported
• AVRO-3467 Use oracle-actions to test with Early Access JDKs
• AVRO-3453 Avrogen Add Generated Code Attribute
• AVRO-3432 Add command line option to skip creation of directories
• AVRO-3411 Add Visual Studio Code Devcontainer support
• AVRO-3388 Implement extra codecs for C# as seperate nuget packages
• AVRO-3265 avrogen generates uncompilable code when namespace ends
  with ".Avro"
• AVRO-3219 Support nullable enum type fields

Java

• AVRO-3531 GenericDatumReader in multithread lead to infinite loop
• AVRO-3482 Reuse MAGIC in DataFileReader
• AVRO-3586 Make Avro Build Reproducible
• AVRO-3441 Automatically register LogicalTypeFactory classes
• AVRO-3375 Add union branch, array index and map key "path"
  information to serialization errors
• AVRO-3374 Fully qualified type reference "ns.int" loses namespace
• AVRO-3294 IDL parsing allows doc comments in strange places
• AVRO-3273 avro-maven-plugin breaks on old versions of Maven
• AVRO-3266 Output stream incompatible with MagicS3GuardCommitter
• AVRO-3243 Lock conflicts when using computeIfAbsent
• AVRO-3120 Support Next Java LTS (Java 17)
• AVRO-2498 UUID generation is not working

Javascript

• AVRO-3489 Replace istanbul with nyc for code coverage
• AVRO-3322 Buffer is not defined in browser environment
• AVRO-3084 Fix JavaScript interop test to work with other languages on CI

Perl

• AVRO-3263 Schema validation warning on invalid schema with a long field

Python

• AVRO-3542 Scale assignment optimization
• AVRO-3521 "Scale" property from decimal object
• AVRO-3380 Byte reading in avro.io does not assert bytes read
• AVRO-3229 validate the default value of an enum field
• AVRO-3218 Pass LogicalType to BytesDecimalSchema

Ruby

• AVRO-3277 Test against Ruby 3.1

Rust

• AVRO-3558 Add a demo crate that shows usage as WebAssembly
• AVRO-3526 Improve resolving Bytes and Fixed from string
• AVRO-3506 Implement Single Object Writer
• AVRO-3507 Implement Single Object Reader
• AVRO-3405 Add API for user-provided metadata to file
• AVRO-3339 Rename crate from avro-rs to apache-avro
• AVRO-3479 Derive Avro Schema macro

Website

• AVRO-2175 Website refactor
• AVRO-3450 Document IDL support in IDEs

Rust

This is the first release that provides the apache-avro crate at crates.io!

JIRA

A list of all JIRA tickets fixed in 1.11.1 could be found here

Language repositories

In addition, language-specific release artifacts are available:

• C#: https://www.nuget.org/packages/Apache.Avro/1.11.1
• Java: https://repo1.maven.org/maven2/org/apache/avro/avro/1.11.1/
• Javascript: https://www.npmjs.com/package/avro-js/v/1.11.1
• Perl: https://metacpan.org/release/Avro
• Python 3: https://pypi.org/project/avro/1.11.1
• Ruby: https://rubygems.org/gems/avro/versions/1.11.1
• Rust: https://crates.io/crates/apache-avro/0.14.0

Thanks to everyone for contributing!

Apache Avro 1.11.0

The Apache Avro community is pleased to announce the release of Avro 1.11.0!

All signed release artifacts, signatures and verification instructions can
be found here: https://avro.apache.org/releases.html

This release includes 120 Jira issues, including some interesting features:

Specification: AVRO-3212 Support documentation tags for FIXED types
C#: AVRO-2961 Support dotnet framework 5.0
C#: AVRO-3225 Prevent memory errors when deserializing untrusted data
C++: AVRO-2923 Logical type corrections
Java: AVRO-2863 Support Avro core on android
Javascript: AVRO-3131 Drop support for node.js 10
Perl: AVRO-3190 Fix error when reading from EOF
Python: AVRO-2906 Improved performance validating deep record data
Python: AVRO-2914 Drop Python 2 support
Python: AVRO-3004 Drop Python 3.5 support
Ruby: AVRO-3108 Drop Ruby 2.5 support

For the first time, the 1.11.0 release includes experimental support for
Rust. Work is continuing on this donated SDK, but we have not versioned and
published official artifacts for this release.

Python: The avro package fully supports Python 3. We will no longer publish a
separate avro-python3 package

And of course upgraded dependencies to latest versions, CVE fixes and more:
https://issues.apache.org/jira/issues/?jql=project%3DAVRO%20AND%20fixVersion%3D1.11.0

The link to all fixed JIRA issues and a brief summary can be found at:
https://github.com/apache/avro/releases/tag/release-1.11.0

In addition, language-specific release artifacts are available:

• C#: https://www.nuget.org/packages/Apache.Avro/1.11.0
• Java: from Maven Central,
• Javascript: https://www.npmjs.com/package/avro-js/v/1.11.0
• Perl: https://metacpan.org/release/Avro
• Python 3: https://pypi.org/project/avro/1.11.0
• Ruby: https://rubygems.org/gems/avro/versions/1.11.0

Thanks to everyone for contributing!

Apache Avro 1.10.2

The Apache Avro community is pleased to announce the release of Avro 1.10.2!

All signed release artifacts, signatures and verification instructions can
be found here: https://avro.apache.org/releases.html

This release includes 31 Jira issues, including some interesting features:

C#: AVRO-3005 Support for large strings
C++: AVRO-3031 Fix for reserved keywords in generated code
Java: AVRO-2471 Fix for timestamp-micros in generated code
Java: AVRO-3060 Support ZSTD level and bufferpool options
Ruby: AVRO-2998 Records with symbol keys validation
Ruby: AVRO-3023 Validate with Ruby 3

Migration notes:
Python: AVRO-2656 The standard avro package supports Python 3, and
the avro-python3 package is in the process of being deprecated. 

And of course upgraded dependencies to latest versions, CVE fixes and more:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2

The link to all fixed JIRA issues and a brief summary can be found at:
https://github.com/apache/avro/releases/tag/release-1.10.2

In addition, language-specific release artifacts are available:

• C#: https://www.nuget.org/packages/Apache.Avro/1.10.2
• Java: from Maven Central,
• Javascript: https://www.npmjs.com/package/avro-js/v/1.10.2
• Perl: https://metacpan.org/release/Avro
• Python 3: https://pypi.org/project/avro/1.10.2/
• Ruby: https://rubygems.org/gems/avro/versions/1.10.2

Thanks to everyone for contributing!

Apache Avro 1.10.1

The Apache Avro community is pleased to announce the release of Avro 1.10.1!

All signed release artifacts, signatures and verification instructions can be
found here: https://avro.apache.org/releases.html

This release includes 33 Jira issues, including some interesting features:

C#: AVRO-2750 Support for enum defaults
C++: AVRO-2891 Expose last sync offset written on DataFileWriter
Java: AVRO-2924 SpecificCompiler add 'LocalDateTime' logical type
Java: AVRO-2937 Expose some missing flags in SpecificCompilerTool
PHP: AVRO-2096 Fixes to missing functions
Ruby: AVRO-2907 Ruby schema.single_object_schema_fingerprint is reversed

Migration notes:
Java: AVRO-2817 Turn off validateDefaults when reading legacy Avro files
Python: AVRO-2656 avro-python package is now the preferred python3 library and
                  avro-python3 is prepared to be deprecated

And of course upgraded dependencies to latest versions, CVE fixes and more: https://issues.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.1

Apache Avro 1.10.0

This release includes 189 Jira issues, including some interesting features:

C#: AVRO-2389 Add Avro serialization for POCO (Reflection)
Java: AVRO-2723 Automatically find defaults on POJO when using reflection
Java: AVRO-2438 Better support for URI and URL types
Perl: AVRO-1461 Distribute Perl module in CPAN
PHP: AVRO-2527 Update to PHP 7.x
Python: AVRO-2387 Type checking added to python
Ruby: AVRO-1740 Support fingerprinting
Ruby: AVRO-2535 Support enum defaults
Ruby: AVRO-2545 Support aliases

Migration notes:
Java: AVRO-2278 Throw an exception when getting a non-existent field from a record
Java: AVRO-2581 Maven plugin generates specific records with private fields
Java: AVRO-2335 Remove Joda Time library
Python: AVRO-2656 avro-python package is now the preferred python3 library and avro-python3 is prepared to be deprecated

And of course upgraded dependencies to latest versions, CVE fixes and more
https://issues.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.0

release-1.10.0-rc2

Avro 1.10.0 rc2 release.

release-1.10.0-rc1

Avro 1.10.0 rc1 release.

Apache Avro 1.9.2

This release includes 71 Jira issues: https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.9.2

• Some bug fixes:
  • C#: AVRO-2606 handle multidimensional arrays of custom types
  • Java: AVRO-2592 Avro decimal fails on some conditions
  • Java: AVRO-2641 Generated code results in java.lang.ClassCastException
  • Java: AVRO-2663 Projection on nested records does not work
  • Python: AVRO-2429 unknown logical types should fall back
• Improvements:
  • Java: AVRO-2247 Improve Java reading performance with a new reader
  • Python: AVRO-2104 Schema normalisation and fingerprint support for Python 3
• Work to unify Python2 and Python3 APIs in preparation for sunset.
• Improved tests
• Improved, more reliable builds.
• Improved readability
• Upgrade dependencies to latest versions, including CVE fixes.
• And more...