CBOR serialization into an OutputStream or WritableByteChannel or externally allocated ByteBuffer

[0x6675636b796f75676974687562]

What is your use-case and why do you need this feature?

When encoding data as JSON, it's possible to write serialized data directly into an OutputStream, via Json.encodeToStream(SerializationStrategy<T>, T, OutputStream).

Yet, with CBOR, only writing to or reading from a ByteArray (byte[] JVM type) seems possible (Cbor.encodeToByteArray(SerializationStrategy<T>, T): ByteArray), and the byte array returned gets allocated by the library itself, can't be supplied externally, nor can it be reused.

This leads to extra allocations and frequent OutOfMemoryError's when encoding large data hierarchies as CBOR, which never happens with JSON.

Describe the solution you'd like

We'd appreciate if any of the following extensions, with the corresponding decodeXyz() counterparts, are eventually implemented:

• Cbor.encodeToStream(SerializationStrategy<T>, T, OutputStream)
• Cbor.encodeToChannel(SerializationStrategy<T>, T, WritableByteChannel)
• Cbor.encodeToBuffer(SerializationStrategy<T>, T, ByteBuffer)

[xiaozhikang0916]

This feature may be commonized as Supporting streaming decoding and encoding with multiplatform.

Personally I like this idea. My team has also faced OOM issue using FastJSON and got solved by using streaming decoding. With this feature in serialization lib, we can have more flexiable control.

We can introduce a new stream-ext lib to support it with help of kotlinx.io, but it would need some refactoring in the core lib and all the formats.