Map.of() cannot be deserialized

[Sam-Kruglov]

I want to use Map.of() in my execution context (primarily for tests). But this gives an error:

val serializer = new Jackson2ExecutionContextStringSerializer();
val out = new ByteArrayOutputStream();
serializer.serialize(Map.of("f", Map.of("f1", "v1")), out);
val json = out.toString();
System.out.println(json);
serializer.deserialize(new ByteArrayInputStream(json.getBytes()));

Output:

{"f":{"@class":"java.util.ImmutableCollections$Map1","f1":"v1"}}
Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve subtype of [map type; class java.util.HashMap, [simple type, class java.lang.String] -> [simple type, class java.lang.Object]]: missing type id property '@class'

The problem is that the typing inside Jackson2ExecutionContextStringSerializer is configured to ObjectMapper.DefaultTyping.NON_FINAL which skips the addition of @class property to the root object for some reason, even though ImmutableCollections$Map1 is final. So, as you can see, only the inner map has it. But if we try to serialize new HashMap<>(Map.of("f", Map.of("f1", "v1"))) then it does include it for the root.
One solution to it is to use ObjectMapper.DefaultTyping.EVERYTHING or enable MapperFeature.USE_BASE_TYPE_AS_DEFAULT_IMPL so that the ObjectMapper doesn't need to check against @class property if it's given the type at compile time.

It knows that it's supposed to be a HashMap from Jackson2ExecutionContextStringSerializer#deserialize:

public Map<String, Object> deserialize(InputStream in) throws IOException {
    TypeReference<HashMap<String,Object>> typeRef = new TypeReference<HashMap<String,Object>>() {};
    return objectMapper.readValue(in, typeRef);
}

So, with enabling the feature, another error comes out:

val serializer = new Jackson2ExecutionContextStringSerializer();
((ObjectMapper) FieldUtils.readField(serializer, "objectMapper", true))
        .enable(MapperFeature.USE_BASE_TYPE_AS_DEFAULT_IMPL);
val out = new ByteArrayOutputStream();
serializer.serialize(Map.of("f", Map.of("f1", "v1")), out);
val json = out.toString();
System.out.println(json);
serializer.deserialize(new ByteArrayInputStream(json.getBytes()));

The output:

{"f":{"@class":"java.util.ImmutableCollections$Map1","f1":"v1"}}
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: The class with java.util.ImmutableCollections$Map1 and name of java.util.ImmutableCollections$Map1 is not trusted. If you believe this class is safe to deserialize, you can add it to the base set of trusted classes at construction time or provide an explicit mapping using Jackson annotations or a custom ObjectMapper. If the serialization is only done by a trusted source, you can also enable default typing. (through reference chain: java.util.HashMap["f"])

I think all java.util.ImmutableCollections$* should be enabled by default.

With adding those, another error comes out:

  val serializer = new Jackson2ExecutionContextStringSerializer(
          "java.util.ImmutableCollections$ListN",
          "java.util.ImmutableCollections$List12",
          "java.util.ImmutableCollections$SubList",
          "java.util.ImmutableCollections$Set12",
          "java.util.ImmutableCollections$SetN",
          "java.util.ImmutableCollections$Map1",
          "java.util.ImmutableCollections$MapN"
  );
  ((ObjectMapper) FieldUtils.readField(serializer, "objectMapper", true))
          .enable(MapperFeature.USE_BASE_TYPE_AS_DEFAULT_IMPL);
  val out = new ByteArrayOutputStream();
  serializer.serialize(Map.of("f", Map.of("f1", "v1")), out);
  val json = out.toString();
  System.out.println(json);
  val res = serializer.deserialize(new ByteArrayInputStream(json.getBytes()));
  System.out.println(res);

The output:

{"f":{"@class":"java.util.ImmutableCollections$Map1","f1":"v1"}}
Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `java.util.ImmutableCollections$Map1` (no Creators, like default constructor, exist): no default constructor found

That's a Jackson issue FasterXML/jackson-databind#2900 that's already been fixed in 2.13.0. So, after updating the Jackson library to 2.13.0 the last code snippet gives the correct output:

{"f":{"@class":"java.util.ImmutableCollections$Map1","f1":"v1"}}
{f={f1=v1}}

Environment
Spring Batch 4.3.3
Spring Boot 2.5.0
JDK Amazon Corretto 11.0.13

[Sam-Kruglov]

Also, the Jackson2ExecutionContextStringSerializer#objectMapper should really be somehow accessible so that I can add my modules or some other settings easily without having to provide my own instance.

I think it would be better if Jackson2ExecutionContextStringSerializer constructor contained the call to findAndRegisterModules().

[fmbenhassine]

Thank you for raising this issue about Jackson. The main branch (v5) is already on Jackson 2.13. So the fix will be in Spring Batch 5.0-M1.

v4.3.x uses Jackson 2.11. We typically only upgrade dependencies to their latest patch versions in a patch release of Spring Batch, so upgrading Jackson to 2.13 in v4.3.x is risky. But you can always override the dependency and use any compatible version of Jackson.

Also, the Jackson2ExecutionContextStringSerializer#objectMapper should really be somehow accessible so that I can add my modules or some other settings easily without having to provide my own instance.

I think the default object mapper is not accessible by design. Making it accessible could open the door to security issues, that's why the default instance is closed for modification.

[fmbenhassine]

Closing this as 5.0.0-M1 has been released and which depends on Jackson 2.13.1.

[Sam-Kruglov]

Hi, wanted to point out a few other things:

1. I mentioned in the issue that it needs all 3 components to work: 1) jackson 2.13 2) enabling java.util.ImmutableCollections$* 3) using USE_BASE_TYPE_AS_DEFAULT_IMPL or DefaultTyping.EVERYTHING. It seems only the first component has been resolved, could you take a look at the other 2?
2. I disagree with "security issues", I'm already using ObjectMapper in my Hibernate configuration, it would be acceptable to use it for batch database as well, it's the same database instance for me. I think Spring Batch should allow me to choose for myself on this one.
3. Even if you won't reconsider point 2, maybe you'll consider adding a call to ObjectMapper#findAndRegisterModules() in Jackson2ExecutionContextStringSerializer's initialization, that would suffice for me. Some of my objects inside job context (inside the Map<String, Object>) are custom and need their own modules to be deserialized properly. Using reflection to make that call right now.

My current solution:

@Bean
@SneakyThrows
public ExecutionContextSerializer customSerializer() {
    val serializer = new Jackson2ExecutionContextStringSerializer(
            "java.util.ImmutableCollections$ListN",
            "java.util.ImmutableCollections$List12",
            "java.util.ImmutableCollections$SubList",
            "java.util.ImmutableCollections$Set12",
            "java.util.ImmutableCollections$SetN",
            "java.util.ImmutableCollections$Map1",
            "java.util.ImmutableCollections$MapN",
            "my.Clazz"
    );
    ((ObjectMapper) FieldUtils.readField(serializer, "objectMapper", true))
            .enable(MapperFeature.USE_BASE_TYPE_AS_DEFAULT_IMPL)
            .findAndRegisterModules();
    return serializer;
}

[fmbenhassine]

I think Spring Batch should allow me to choose for myself on this one.

Spring Batch provides a setter of the object mapper in order to use your custom one. I'm not sure I understand why you need reflection to set it on the serializer. The following should cover the 3 points you mentioned:

@Bean
@SneakyThrows
public ExecutionContextSerializer customSerializer() {
    Jackson2ExecutionContextStringSerializer serializer = new Jackson2ExecutionContextStringSerializer(
            "java.util.ImmutableCollections$ListN",
            "java.util.ImmutableCollections$List12",
            "java.util.ImmutableCollections$SubList",
            "java.util.ImmutableCollections$Set12",
            "java.util.ImmutableCollections$SetN",
            "java.util.ImmutableCollections$Map1",
            "java.util.ImmutableCollections$MapN",
            "my.Clazz"
    );
    ObjectMapper objectMapper = new ObjectMapper()
        .enable(MapperFeature.USE_BASE_TYPE_AS_DEFAULT_IMPL)
        .findAndRegisterModules();
    serializer.setObjectMapper(objectMapper);
    return serializer;
}