AvroConvert

Rapid Avro serializer for C# .NET

Docs

Avro format combines readability of JSON and data compression of binary serialization.

Apache Wiki

Apache Avro format documentation

AvroConvert Documentation

First steps with Avro in the .NET article

Benchmark and Avro API article

Benefits

Introducing Avro to the projects brings three main benefits:

• Reduction of data size and storage cost
• Decrease of the communication time and the network traffic between microservices
• Increased security - the data is not visible in plain text format

Features

	AvroConvert	Apache.Avro	Newtonsoft.Json
Rapid serialization	✔️	✔️	✔️
Easy to use	✔️	❌	✔️
Built-in compression	✔️	✔️	❌
Low memory allocation	✔️	✔️	✔️
Support for C# data structures: Dictionary, List, DateTime...	✔️	❌	✔️
Support for compression codecs	Deflate Snappy GZip Brotli	Deflate	❌
Readable schema of data structure	✔️	✔️	✔️
Data encryption	✔️	✔️	❌

Full Changelog

Benchmark

Results of BenchmarkDotNet:

Converter	Request Time [ms]	Allocated Memory [MB]	Compressed Size [kB]
Json	672.3	52.23	6044
Avro	384.7	76.58	2623
Json_Gzip	264.1	88.32	514
Avro_Gzip	181.2	75.05	104
Json_Brotli	222.5	86.15	31
Avro_Brotli	193.5	74.75	31

Article describing Avro format specification and benchmark methodology: https://www.c-sharpcorner.com/blogs/avro-rest-api-as-the-evolution-of-json-based-communication-between-mic

Conclusion:
Using Avro for communication between your services significantly reduces data size and network traffic. Additionally choosing encoding (compression algorithm) can improve the results even further.

Code samples

• Serialization

 byte[] avroObject = AvroConvert.Serialize(object yourObject);

• Deserialization

CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);

• Read schema from Avro object

string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)

• Deserialization of large collection of Avro objects one by one

using (var reader = AvroConvert.OpenDeserializer<CustomClass>(new MemoryStream(avroObject)))
{
    while (reader.HasNext())
    {
        var item = reader.ReadNext();
        // process item
    }
}

• Generation of C# models from Avro file or schema

  string resultModel = AvroConvert.GenerateModel(avroObject);

• Conversion of Avro to JSON directly

  var resultJson = AvroConvert.Avro2Json(avroObject);

Full Documentation

Related Work

1. AvroConvertOnline - online Avro Schema to C# model converter

2. - Library containing functionalities, which enable communication between microservices via Http using Avro data format

3. - Library containing components needed for Confluent Kafka integration

License

The project is CC BY-NC-SA 3.0 licensed.
For commercial purposes purchase AvroConvert on website - Xabe.net

Contribution

We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.