Reading Avro with specified reader and writer schemas

[pszymczyk]

Hi

In origin avro-tools when I read some data I can specify reader and writer schemas. It is related with backward/forward compatibility:

SpecificDatumReader r = new SpecificDatumReader(writer, reader);
BinaryDecoder decoder = DecoderFactory.get().directBinaryDecoder(inputStream, null);
return r.read(null, decoder);

Is it possible to specify writer and reader schema in AvroMapper?

[cowtowncoder]

Not currently. I am not sure what I think of ability to do this, although if suitable API could be specified this might make sense.

[pszymczyk]

My case is that in our company we have implemented some generic SerDe library built on the top of origin Avro tools. Now we'd like to migrate to Jackson but we can't cause it does not expose method with reader and writer schema :(.

[cowtowncoder]

@pszymczyk I guess my question there is as to value of separate schemas. Since Jackson data-binding is somewhat more flexible, it often should be enough to just use the writer schema that was used for generation? And since this is always required to be present, value of second schema (to rename fields, essentially) may not be that high.

On API: the main challenge is just that of how to pass it via general Jackson API. I suspect that if it is necessary, it would be possible to modify AvroSchema wrapper (or whatever the name was) to allow secondary schema, and so maybe support this quite seamlessly?
Avro parser and generators must know about handling of AvroSchema anyway, so this might be relatively simple change actually.

If you happened to have time and interest, I would be happy to help getting it integrated via PR.

[cowtowncoder]

@panchenko do you happen to have a simple case of reader+writer schema that I could use? I think I should be able to implement this quite easily after all.

[cowtowncoder]

This is now implemented for upcoming 2.8.7. Usage is via AvroSchema, with something like:

AvroSchema writerSchema = mapper.schemaFrom(src);
AvroSchema resolving = writerSchema.withReaderSchema(
     mapper.schemaFrom(srcForReaderSchema));

and then just using given schema.

Note on performance: while there may be some overhead for schema evolution there should not be significant differences during parsing -- most of the work is done during construction of "resolving" schema definition. Because of this it is strongly recommended that schemas are reused: instances are thread-safe so they can be safely shared and reused across threads.