Problem with replacement of @DBRef to @DocumentReference

[ALGA0887]

Hello!

I would like to describe a problem with replacement of annotation @DBRef to annotation @DocumentReference in case old data in MongoDB exist. For this purpose I have created a project with reproducing of this problem: https://github.com/ALGA0887/reference.git.

So lets imagine the following class in previous releases of some application:

@Data
@AllArgsConstructor
@Document(collection = "guests")
public class Guest {
    @Id
    private String id;
    @Field
    private String name;
    @DBRef
    private List<OrderItem> orderItems;
}

Here there is a @DBRef to the list of OrderItem:

@Data
@AllArgsConstructor
@Document(collection = "order-items")
public class OrderItem {
    @Id
    private String id;
    @Field
    private String name;
    @Field
    private List<String> guestIds;
}

OrderItem itself also contains information about guestIds.
So in previous releases guests were saved in MongoDB with DBRef to orderItems.

In current release it was decided not to have a field orderItems in guests collection since it is overhead. And it was planned to change the annotation @DBRef to annotation @DocumentReference in the following way:

@Data
@Document(collection = "guests")
public class Guest {
    @Id
    private String id;
    @Field
    private String name;
    @ReadOnlyProperty
    @DocumentReference(lookup = "{'guestIds':?#{#self._id}}")
    private List<OrderItem> orderItems;
    public Guest(String id, String name) {
        this.id = id;
        this.name = name;
    }
}

For new records in guests such changes work as expected (test com.alga.reference.ReferenceApplicationTests#testLoadGuestWithoutDbRef). But old records can't be read with the following exception:

org.springframework.expression.spel.SpelEvaluationException: EL1008E: Property or field '_id' cannot be found on object of type 'com.mongodb.DBRef' - maybe not public or not valid?

	at org.springframework.expression.spel.ast.PropertyOrFieldReference.readProperty(PropertyOrFieldReference.java:228)
	at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:111)
	at org.springframework.expression.spel.ast.PropertyOrFieldReference$AccessorValueRef.getValue(PropertyOrFieldReference.java:416)
	at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:98)
	at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:119)
	at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:309)
	at org.springframework.data.mongodb.util.json.EvaluationContextExpressionEvaluator.evaluateExpression(EvaluationContextExpressionEvaluator.java:69)
	at org.springframework.data.mongodb.util.json.ParameterBindingContext.evaluateExpression(ParameterBindingContext.java:115)
	at org.springframework.data.mongodb.util.json.ParameterBindingJsonReader.evaluateExpression(ParameterBindingJsonReader.java:535)
	at org.springframework.data.mongodb.util.json.ParameterBindingJsonReader.bindableValueFor(ParameterBindingJsonReader.java:395)
	at org.springframework.data.mongodb.util.json.ParameterBindingJsonReader.readBsonType(ParameterBindingJsonReader.java:300)
	at org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:237)
	at org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:182)
	at org.springframework.data.mongodb.core.convert.ReferenceLookupDelegate.computeFilter(ReferenceLookupDelegate.java:281)
	at org.springframework.data.mongodb.core.convert.ReferenceLookupDelegate.readReference(ReferenceLookupDelegate.java:109)
	at org.springframework.data.mongodb.core.convert.DefaultReferenceResolver.resolveReference(DefaultReferenceResolver.java:76)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readAssociation(MappingMongoConverter.java:655)
...

So my question is why DBRef is taken into account in lookup expression? Is it bug in reference resolving?

In my project I have tried the following workaround and it works:

@Configuration
public class MongoConfig {
    @Bean
    @Profile("workaround")
    public MappingMongoConverter mappingMongoConverter(MongoDatabaseFactory factory, MongoMappingContext mappingContext) {
        // GuestDbRefResolver is a workaround for loading of old data: guests with DBRef-s.
        // With DefaultDbRefResolver loading of old data is failed with SpelEvaluationException and the following message:
        //      "EL1008E: Property or field '_id' cannot be found on object of type 'com.mongodb.DBRef' - maybe not public or not valid?".
        // To check it comment @ActiveProfiles(value = "workaround") on a test class com.alga.reference.ReferenceApplicationTests
        // and run test com.alga.reference.ReferenceApplicationTests.testLoadGuestWithDbRef
        DbRefResolver dbRefResolver = new GuestDbRefResolver(factory);
        return new MappingMongoConverter(dbRefResolver, mappingContext);
    }
}

public class GuestDbRefResolver extends DefaultDbRefResolver {
    public GuestDbRefResolver(MongoDatabaseFactory mongoDbFactory) {
        super(mongoDbFactory);
    }
    @SneakyThrows
    @Override
    public Object resolveReference(MongoPersistentProperty property, Object source, ReferenceLookupDelegate referenceLookupDelegate, MongoEntityReader entityReader) {
        Object resultSource = source;
        if (source instanceof DocumentReferenceSource drs && drs.getTargetSource() != null && "orderItems".equals(property.getFieldName())) {
            Class<?> ownerClass = property.getOwner().getTypeInformation().getType();
            Class<?> targetClass = property.getAssociationTargetType();
            if (ownerClass == Guest.class && targetClass == OrderItem.class) {
                // Why constructor for DocumentReferenceSource is not public?
                Constructor<DocumentReferenceSource> drsc = DocumentReferenceSource.class.getDeclaredConstructor(Object.class, Object.class);
                drsc.setAccessible(true);
                resultSource = drsc.newInstance(drs.getSelf(), null);
            }
        }
        return super.resolveReference(property, resultSource, referenceLookupDelegate, entityReader);
    }
}

But if we imagine that migration of data is impossible (I mean remove DBRef for old records in guests) then how we can apply such changes without any workaround at all?

To reproduce a problem just comment @ActiveProfiles(value = "workaround") in ReferenceApplicationTests and run test testLoadGuestWithDbRef.

@RunWith(SpringRunner.class)
@SpringBootTest(classes = ReferenceApplication.class)
//@ActiveProfiles(value = "workaround")
public class ReferenceApplicationTests {

    ...    

    @Test
    public void testLoadGuestWithDbRef() {
        ...
    }
}

[christophstrobl]

Thank you for reaching out!
@DocumentReference is no drop in replacement for @DBRef, but an alternative approach of storing references between documents. I'm not aware of any section in the reference documentation that would promote the given scenario to work out of the box. If so, please let us know so we can update that part to be more clear.

[ALGA0887]

Hello @christophstrobl!

Thank you for your answer!
Yes you are right: there is nothing directly about such replacement in the documentation.
Nevertheless it is not expected behavior that processing of such combination of annotations @DocumentReference and @ReadOnlyProperty on some field leads to loading of any data (in our case it is DBRef but in general it could be anything) from MongoDB during evaluation of Spel expression. With spring-data our schema is specified in java code and java code should have higher priority than data in MongoDB. In our code this field is not DBRef and should be ignored by default.

https://docs.spring.io/spring-data/mongodb/docs/current-SNAPSHOT/reference/html/#mapping-usage.document-references
It is also possible to model relational style One-To-Many references using a combination of @ReadonlyProperty and @DocumentReference. This approach allows link types without storing the linking values within the owning document but rather on the referencing document as shown in the example below.

So there is expectation after reading of documentation above that combination of @ReadonlyProperty and @DocumentReference tells Mapping Framework just ignore data in this field in MongoDB. If we don't store anything in the field, why it is necessary to read anything from it? How we can ignore existing data in MongoDB then?

[ALGA0887]

Hello @christophstrobl!

Could you please share plans regarding this issue taking into account my last comment?