Custom Serializer - Access Annotation Info

[austinarbor]

Is your feature request related to a problem? Please describe.

When writing a custom serializer, I'd like to be able to know the Jackson annotations that were on the entity being serialized. Serializing fields may be be intensive or cause recursion, so if the caller has specified to ignore that field, I would like to know before I try and serialize it

Describe the solution you'd like

Either JsonGenerator or SerializerProvider (or some other static method) should provide a lookup mechanism to get the Jackson annotations related to the current object being serialized

Usage example

data class Parent(
  val name: String,
  @JsonIgnoreProperties("parent") 
  val child: Child
)
data class Child(
  val name: String,
  val parent: Parent,
)

class ChildSerializer : StdSerializer<Child>(Child::class.java) {
  override fun serialize(obj: Child, gen: JsonGenerator, provider: SerializerProvider) {
     gen.writeStartObject()
     gen.writeFieldName("name")
     gen.writeString(obj.name)
     // if i am serializing a child that is accessed from within a parent object
     // that should not be serialized because of @JsonIgnoreProperties
     // but if I am serializing a Child directly, the parent should be serialized
     gen.writeEndObject()
  }
}

Additional context

No response

[pjfanning]

Can you not use Java reflection APIs to get the annotations from the class and maybe its fields? I don't quite see why Jackson needs to have an API for this when Java provides one out of the box.

[austinarbor]

@pjfanning if I am serializing Child, how do I know if I am serializing it as a member of Parent, or as its own independent entity?

[pjfanning]

@pjfanning if I am serializing Child, how do I know if I am serializing it as a member of Parent, or as its own independent entity?

Serializers are usually registered for a particular class. For instance, you have class ChildSerializer : StdSerializer<Child> - what other class is meant to be handled if you explicitly say you want to only serialize Child instances.

Surely, Child has a reference to Parent. override fun serialize(obj: Child, -- can't you check if obj is null or not null -- and then check if child.parent is null or not null?

[austinarbor]

my original example has the use case

data class Parent(val name: String) {
  @JsonIgnoreProperties("parent") 
  var child: Child? = null
}
data class Child(
  val name: String,
  @JsonIgnoreProperties("child")
  val parent: Parent,
) 

My serializer will handle only serializing Child in both cases. However, in the case of parent -> child , child.parent should not be serialized. In the case of child, child.parent should be. Whether child.parent is null or not isn't a good enough indicator if it should be serialized..

val parent = Parent("parent")
val child = Child("child", parent)
parent.child = child

// if I serialize parent.child.parent i get infinite recursion
mapper.writeValueAsString(parent)
// if i serialize child.parent.child i get infinite recursion
mapper.writeValueAsString(child)

when serializing the parent, I should get

{ "name" : "parent", "child": { "name": "child" }}

and when serializing the child I should get

{ "name" : "child", "parent": { "name": "parent" }}

[JooHyukKim]

@austinarbor If infintie recursion is your problem, I think it is resolved in at least 2.17 version.
Please refer below for reproducible test that runs and passes against Jackson Kotlin module version 2.17.

Note that serializing child outputs to "{\"name\":\"child\"}".

class `4501Test` {

    data class Parent(val name: String) {
        @JsonIgnoreProperties("parent")
        var child: Child? = null
    }

    data class Child(
        val name: String,
        @JsonIgnoreProperties("child")
        val parent: Parent,
    )

    class ChildSerializer : StdSerializer<Child>(Child::class.java) {
        override fun serialize(obj: Child, gen: JsonGenerator, provider: SerializerProvider) {
            gen.writeStartObject()
            gen.writeFieldName("name")
            gen.writeString(obj.name)
            gen.writeEndObject()
        }
    }

    @Test
    fun `reproduce infinite recursion`() {
        // Given
        val simpleModule = SimpleModule().addSerializer(Child::class.java, ChildSerializer())
        val mapper = jacksonObjectMapper()
            .registerModules(simpleModule)

        val parent = Parent("parent")
        val child = Child("child", parent)
        parent.child = child

        // When & Then
        assertEquals(
            "{\"name\":\"parent\",\"child\":{\"name\":\"child\"}}",
            mapper.writeValueAsString(parent)
        )
        assertEquals(
            "{\"name\":\"child\"}",
            mapper.writeValueAsString(child)
        )
    }

}

[austinarbor]

@JooHyukKim your ChildSerializer is incorrect, it should be

class ChildSerializer : StdSerializer<Child>(Child::class.java) {
        override fun serialize(obj: Child, gen: JsonGenerator, provider: SerializerProvider) {
            gen.writeStartObject()
            gen.writeFieldName("name")
            gen.writeString(obj.name)
            gen.writeFieldName("parent")
            gen.writeObject(obj.parent)
            gen.writeEndObject()
        }
    }

Running that, you get

Document nesting depth (1002) exceeds the maximum allowed (1000, from StreamWriteConstraints.getMaxNestingDepth()`) (through reference chain: ...

[yawkat]

@austinarbor you can implement a ContextualSerializer to access annotations on the property

[cowtowncoder]

What @yawkat said -- Jackson design is such that ALL annotation access must come from "static" part of processing, when constructing and initializing (de)serializers and other handlers: this information is not (and will not) ever be passed during more dynamic serialization/deserialization code path.
Serializers and deserializers can however (and some do) store necessary information during construction/initialization, and ContextSerializers createContextual() callback is where information is passed.

So information is not and will not be passed through serialize() method. This is by design.

[austinarbor]

@yawkat @cowtowncoder thanks! ContextualSerializer seems to be working well - I did have a little bit of trouble figuring out the best way to "propagate" the annotations to downstream serializers. For example, I am working with Jetbrains' exposed ORM which has a LazySizedCollection (their implementation of a list). I was trying to replicate the behavior of how the @JsonIgnoreProperties annotation gets applied to a list of objects, for example

class MyObject {
  @JsonIgnoreProperties("someProperty") // the annotation is applied to each item in the list
  val others : List<OtherObject> = listOf()
}

I wasn't really able to replicate that functionality fully, is there a better approach to use than provider.findValueSerializer below?

class LazySizedCollectionSerializer(private val bean: BeanProperty? = null) :
    StdSerializer<LazySizedCollection<*>>(
        LazySizedCollection::class.java,
    ),
    ContextualSerializer {
  override fun serialize(
      value: LazySizedCollection<*>,
      gen: JsonGenerator,
      provider: SerializerProvider,
  ) {
    gen.writeStartArray()
    // if the value is loaded from the database already, serialize it
    // otherwise just write an empty array
    if (value.isLoaded()) {
      for (item in value) {
        if (item == null) {
          gen.writeNull()
        } else {
          // for some reason, the calls to _other_ createContextual custom serializers
          // all were passing a null bean property, even with this below
          val ser = provider.findValueSerializer(item::class.java, bean)
          ser.serialize(item, gen, provider)
        }
      }
    }
    gen.writeEndArray()
  }

  override fun createContextual(
      prov: SerializerProvider?,
      property: BeanProperty?,
  ): JsonSerializer<*> {
    // the bean property here contains the JsonIgnoreProperties annotation information
    return LazySizedCollectionSerializer(property)
  }
}

[cowtowncoder]

Hmmh. Yeah, that is challenging thing to do; given that handling of elements (applying annotations to collection elements), it might make sense to instead use BeanSerializerModifier.

And implementing custom List serializer it might make sense to extend a more advanced base class than StdSerializer: f.ex

src/main/java/com/fasterxml/jackson/databind/ser/impl/IndexedListSerializer.java

extends AsArraySerializerBase.

Hopefully some combination works.

[austinarbor]

The BeanSerializerModifier idea actually helped a lot because it gave me access to the BeanProperties from the original bean analysis. I think I was able to get something working, does it look like I'm doing anything horribly wrong?

class SerializerModifier : BeanSerializerModifier() {
  override fun modifySerializer(
      config: SerializationConfig,
      beanDesc: BeanDescription,
      serializer: JsonSerializer<*>,
  ): JsonSerializer<*> {
    if (beanDesc.beanClass.kotlin.isSubclassOf(Entity::class)) {
      return EntitySerializer(serializer as JsonSerializer<Any>)
    }

    return serializer
  }
}

class EntitySerializer(private val ser: JsonSerializer<Any>, private val bean: BeanProperty? = null) 
  : StdSerializer<Entity<*>>(Entity::class.java), ContextualSerializer {
  
  private val ignoredProperties: Set<String>

  init {
    val ignored = mutableSetOf<String>()
    if (bean != null) {
      // the javadoc says to use the AnnotationInspector, but the ignored properties never change so 
      // i thought it would be more efficient to calculate them once up front
      bean.getAnnotation(JsonIgnoreProperties::class.java)?.value?.forEach { ignored.add(it) }
      bean.getContextAnnotation(JsonIgnoreProperties::class.java)?.value?.forEach {
        ignored.add(it)
      }
    }
    ignoredProperties = ignored
  }

  override fun serialize(entity: Entity<*>, gen: JsonGenerator, provider: SerializerProvider) {
    gen.writeStartObject()
    // right now this code looks like it could be replaced by the default serializer,
    // but i am excluding some custom logic for brevity. i just want to demonstrate how
    // i am determining the ignored properties
    ser.properties()
        .asSequence()
        .filterNot { it.name in ignoredProperties }
        .forEach { prop ->
          gen.writeFieldName(prop.name)
          val v = prop.member.getValue(entity)
          if (v == null) {
            gen.writeNull()
          } else {
            val valueSerializer = provider.findValueSerializer(v::class.java, prop)
            valueSerializer.serialize(v, gen, provider)
          }
        }
    gen.writeEndObject()
  }

  override fun createContextual(prov: SerializerProvider, property: BeanProperty?): JsonSerializer<*> {
    return EntitySerializer(this.ser, property)
  }
}

class LazySizedCollectionSerializer(private val bean: BeanProperty? = null) :
    StdSerializer<LazySizedCollection<*>>(LazySizedCollection::class.java), ContextualSerializer {

  override fun serialize(
      value: LazySizedCollection<*>,
      gen: JsonGenerator,
      provider: SerializerProvider,
  ) {
    if (value.isLoaded()) {
      val ser = provider.findValueSerializer(List::class.java, bean)
      ser.serialize(value.wrapper, gen, provider)
    } else {
      // value was not specified to be eagerly fetched from the database so write null
      gen.writeNull()
    }
  }

  override fun createContextual(provider: SerializerProvider?,  property: BeanProperty?): JsonSerializer<*> {
    return LazySizedCollectionSerializer(property)
  }
}

[cowtowncoder]

That looks fine overall, except this:

bean.getContextAnnotation(JsonIgnoreProperties::class.java)

is probably not what you want: it'd be for class annotations of class that has property with List value. While such defaulting is done for other uses, it seems incorrect here.

Actually... another thing worth considering is that BeanDescription has lots of useful accessors to use... and I'd thing getIgnoredPropertyNames() is what you'd want? It is based on both Annotations and Config overrides. So if that works, it's preferable over explicit annotation access.

And from performance perspective, having to dynamically call findValueSerializer() for serialize() can be quite inefficient. Not sure if that can be avoided, although default BeanSerializer tries resolving these during createContextual() (or maybe resolve()), and even in the case of having to dynamically resolve serializers does lazy-caching using PropertySerializerMap. But that gets rather complicated... but if you could reuse standard value serializers, that'd be supported out of the box.

Oh, one more thing: JsonSerializer has "mutant factory method":

public JsonSerializer<?> withIgnoredProperties(Set<String> ignoredProperties)

so you can apply ignorals (serializers for which these do not apply, like scalar serializers, simply do return this without error).

Anyway, I hope some of above is useful. Good progress so far! :)

[austinarbor]

Hmm... getIgnoredPropertyNames seems to always be empty in the BeanDescription, not sure why it's missing the values

Agreed this probably isn't the most performant - I opened a ticket on the Exposed YouTrack to add a feature which would let me optimize a bit more

The only reason I need the custom serializer is because I need to wrap the getValue call in a try/catch and ignore a specific exception and write null when it happens.

Exposed currently doesn't have a way to checking if a 1:1 association has been fetched yet, and if you try and fetch the association after the transaction is closed you get an exception. It's not pretty, but at the moment it's the only way to get it done (that I know of). If there's an easier way to implement writing a null when a specific exception occurs on the getter invocation I would love to hear it

 ser.properties()
        .asSequence()
        .filterNot { it.name in ignoredProperties }
        .forEach { prop ->
          gen.writeFieldName(prop.name)
          try {
            val v = prop.member.getValue(entity)
            if (v == null) {
              gen.writeNull()
            } else {
              val valueSerializer = provider.findValueSerializer(v::class.java, prop)
              valueSerializer.serialize(v, gen, provider)
            }
          } catch(e: IllegalArgumentException) {
            // if we get an IllegalArgumentException here with a certain underlying cause, it means
            // the entity's association was not fetched and we want to just write null to the json
            gen.writeNull()
         }
        }

I wonder...can I provide a custom Introspector for one class only that provides BeanProperty with a wrapped getValue implementation? I could put the try/catch in the custom implementation and avoid the custom serializer entirely

[cowtowncoder]

There is no way to change AnnotationIntrospector configured for an ObjectMapper after construction, or to override it in any way.

Odd that BeanDescription has empty/missing one; would be good to know if a test could be isolated to show the problem (without all the other surrounding code).
But in short term it is what it is -- depending, it could be that annotation only visible to List serializer and you'd need it for element within.