Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@JsonNaming does not work with Java 16 record types #3102

Closed
dain opened this issue Apr 2, 2021 · 16 comments
Closed

@JsonNaming does not work with Java 16 record types #3102

dain opened this issue Apr 2, 2021 · 16 comments
Labels
Record Issue related to JDK17 java.lang.Record support

Comments

@dain
Copy link

dain commented Apr 2, 2021

Describe the bug
JsonNaming does not seem to be applied to Java 16 record types for deserialization.

Version information
2.12.2

To Reproduce
Simple unit test:

public class RecordTest
{
    @JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy.class)
    public record TestRecord(int id, String toSnakeCase) {}

    @Test
    public void testSerializeJsonTestRecord()
            throws JsonProcessingException
    {
        JsonMapper jsonMapper = new JsonMapper();

        TestRecord expectedRecord = new TestRecord(123, "Bob");

        String actualJson = jsonMapper.writeValueAsString(expectedRecord);
        assertEquals("{\"id\":123,\"to_snake_case\":\"Bob\"}", actualJson);

        TestRecord actualValue = jsonMapper.readValue("{\"id\":123,\"to_snake_case\":\"Bob\"}", TestRecord.class);
        assertEquals(new TestRecord(123, "Bob"), actualValue);
    }
}

Failure stack:


com.fasterxml.jackson.databind.JsonMappingException: Can not set final java.lang.String field io.starburst.stargate.portal.security.client.RecordTest$TestRecord.toSnakeCase to java.lang.String

	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:274)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:623)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:611)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:634)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.set(FieldProperty.java:193)
	at com.fasterxml.jackson.databind.deser.impl.PropertyValue$Regular.assign(PropertyValue.java:62)
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:211)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:520)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3548)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3516)
	at io.starburst.stargate.portal.security.client.RecordTest.testSerializeJsonTestRecord(RecordTest.java:29)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: java.lang.IllegalAccessException: Can not set final java.lang.String field io.starburst.stargate.portal.security.client.RecordTest$TestRecord.toSnakeCase to java.lang.String
	at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwFinalFieldIllegalAccessException(UnsafeFieldAccessorImpl.java:76)
	at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwFinalFieldIllegalAccessException(UnsafeFieldAccessorImpl.java:80)
	at java.base/jdk.internal.reflect.UnsafeQualifiedObjectFieldAccessorImpl.set(UnsafeQualifiedObjectFieldAccessorImpl.java:79)
	at java.base/java.lang.reflect.Field.set(Field.java:793)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.set(FieldProperty.java:190)
	... 74 more
@dain dain added the to-evaluate Issue that has been received but not yet evaluated label Apr 2, 2021
@cowtowncoder cowtowncoder added the Record Issue related to JDK17 java.lang.Record support label Apr 2, 2021
@cowtowncoder
Copy link
Member

Yes, alas, I think this is same as #2992 (and probably #2974). The problem is that linking of constructor properties is failing for some reason and so only mutator attached to underlying Field gets properly renamed.

I plan on finally tackling bigger property introspection change for 2.13 (to solve the root cause), although if someone can figure out a patch for 2.12.x (with more targeted fix) that would be great too.

@dain
Copy link
Author

dain commented Apr 2, 2021

I tried following the code in a debugger, and I saw the record constructor being processed and registered, with the raw record field names (the constructor parameter names). It looked like it was just missing the processing of the rename strategy. I couldn't follow the rest of the code, but it appears that it also registers the "renamed" the raw private fields inside the record class. Then during deserialization, it sees a snake case field, and since the constructor parameters aren't processed, it tries to set the field directly, which fails.

@cowtowncoder
Copy link
Member

Correct. The problem is due to Creator methods (and properties that are found via them) are handled separately due to historical reasons. If I remember this correctly if (but only if) constructor is explicitly annotated with @JsonCreator, its parameters will be added early enough to be accessible for renaming purposes.

So I think one work around, until fix, would be to add explicit, otherwise unnecessary declaration of Record constructor.

I guess if above is correct, then a smaller fix within Jackson 2.12 would be to indicate that the main Record constructor was explicitly annotated -- one reason this wasn't done is because of possibility that user might designate an alternate constructor. But maybe it'd be possible to detect the case of no explicit annotations.

@dain
Copy link
Author

dain commented Apr 3, 2021

I tried adding an explicit constructor with @JsonCreator and that doesn't work (same error). Think the issue it the new special method to discover the record constructor. Maybe it messes up the explicit constructor.

The only workaround I could find was the obvjous to declare the property mapping manually:

public record TestRecord(int id, @JsonProperty("to_snake_case") String toSnakeCase) {}

@cowtowncoder
Copy link
Member

Hmmh. Ok, I better add a new test for this fail, just in case. Explicit @JsonCreator should connect things.

@cowtowncoder
Copy link
Member

Yes, I can reproduce the issue. Also odd: exception message claims there are no known properties -- I get bit different one with JDK 14, will need to install JDK 16 to see if I can reproduce exception shown here.

@cowtowncoder
Copy link
Member

Ok. The odd "0 properties" was due to local changes (wrt attempt to prevent use of setters which has side effect of dropping properties); after removing that, I can reproduce the issue with JDK 16. @JsonCreator has no effect.

I think I must have misremembered the logic: it must be that explicit naming of constructor properties is the only thing that establishes links, not constructor annotation itself. And obviously having to use that prevents any benefit of applying naming convention.

cowtowncoder added a commit that referenced this issue Apr 3, 2021
@cowtowncoder
Copy link
Member

cowtowncoder commented Apr 3, 2021

Odd. Trying to add a reproduction with regular POJO, it looks like combination of @JsonCreator and implicit names (from jackson-module-parameter-names) actually works. That is... unexpected.

I think this is because Record handling short-circuits otherwise default implicit-name access (since we know how to access them more reliably). So it might be possible to inject implicit names. That would solve this particular issue, although would not help the main use case (of not requiring bogus constructor to add @JsonCreator to). But would perhaps be doable for 2.12.x.

@cowtowncoder cowtowncoder removed the to-evaluate Issue that has been received but not yet evaluated label Jul 15, 2021
@spinscale
Copy link

I just ran into this without using any annotations, by configuring the jackson mapper like this (suppose this is well known as the annotation is doing the same, but I figured I'll leave the test case here):

    @Test
    public void testSerializeDeserializeSnakeCaseRecord() throws JsonProcessingException {
        final ObjectMapper mapper = new ObjectMapper()
                .setPropertyNamingStrategy(PropertyNamingStrategies.SNAKE_CASE);

        User user = new User("Obama", "Barack");

        // serialization
        final String output = mapper.writeValueAsString(user);
        assertThat(output).contains("\"name\":\"Obama\"");
        assertThat(output).contains("\"first_name\":\"Barack\"");

        // deserialization
        final User readUser = mapper.readValue(output, User.class);
        assertThat(readUser.firstName()).isEqualTo("Barack");
        assertThat(readUser.name()).isEqualTo("Obama");
    }

    public static record User(String name, /*@JsonProperty("first_name")*/ String firstName) {
    }

Works fine, when using the @JsonProperty in the record, so a safe workaround.

@pjfanning
Copy link
Member

@cowtowncoder would it be possible to consider the change in FasterXML/jackson-future-ideas#61 (comment) for v2.14?

@cowtowncoder
Copy link
Member

@pjfanning Yes. I haven't had time to follow on those ideas but yes I'd be +1 for inclusion.

@pjfanning
Copy link
Member

I think it might be possible to sort out the issue with Record deserialization when naming strategies are involved by treating Records as a special case and changing it so that the properties are evaluated before the Record constructor is called. When there are no naming strategies, this already happens. When there is a naming strategy, the Record is constructed with null values and the code tries to use setters after the fact to set the right values (and refelction doesn't allow this for Records).

@cowtowncoder
Copy link
Member

@pjfanning At the high level, yes. Records should be handled in (more) special way. But aside from logics there is an existing problem wherein Creator properties (constructor detection) and "regular" property detection are done in somewhat wrong order and that is the big thing I hope to rewrite. Smaller tweaks to the existing implementation are much more complicated unfortunately; but rewrite is not a trivial task either.

Renaming, in particular, was (if I remember correctly) victim of Creator not being found and linked early enough to have the logical creator properties be renamed; thereby only leaving Field-based property accessors in use (unliked with wrongly named creator properties).

@agentgt
Copy link

agentgt commented Aug 18, 2022

Hopefully this is related and might be helpful to others.

They only way I can reliably get records to work is by turning off all Auto Detection (@JsonAutoDetect) and manually doing the @JsonProperty for each component.

Luckily you can make combined annotations that will turn off the auto detection.

@Target({ ElementType.ANNOTATION_TYPE, ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
@JacksonAnnotationsInside
@JsonAutoDetect(//
		getterVisibility = JsonAutoDetect.Visibility.NONE, //
		isGetterVisibility = JsonAutoDetect.Visibility.NONE, //
		setterVisibility = JsonAutoDetect.Visibility.NONE, //
		fieldVisibility = JsonAutoDetect.Visibility.NONE)
@JsonInclude(Include.NON_DEFAULT)
public @interface MyJsonRecord {

}

Really you just have to turn off getterVisibility. The problem with getterVisibility turned on is if your record implements interfaces and you have methods with the same getter name as an accessor (e.g. name() and getName() ) there is no way to ignore it. If you put say @JsonIgnore on say getName() then the name() record accessor will be ignored! If you do not ignore getName() Jackson will call it.

I guess what I'm saying is OOB auto detect for records seems broken and there needs to be way to define accessors besides getters and fields.

It seems like out of the box for records jackson should only care about the record components.
Maybe I'm missing something? Should I file this as a separate bug?

@cowtowncoder
Copy link
Member

@agentgt Yes, there is one fundamental implementation problem that causes this -- it is not mismatch between non-getter and getter as much as missing linkage between "Creator" parameters and "regular" property accessors (getters, setters). This is probably why annotations do not get properly considered for all accessors of a single logical property.

I am hoping to eventually fix this, but unfortunately the change is to rewrite property introspection and I just haven't had time to allocate for this work. On plus side we have tests to verify fix, as well as guard against various regressions.

In the meantime I appreciate your sharing of workarounds so others can use them too.

@cowtowncoder
Copy link
Member

Ok I am pretty sure this is same as #2992 and since that is earlier, will close this one as duplicate.
So the solution for same will be marked as fix for #2992 when we finally get there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Record Issue related to JDK17 java.lang.Record support
Projects
None yet
Development

No branches or pull requests

5 participants