Performance question (ReflectionUtils.getDeclaredMethodOrNull)

[mikoet]

Hey, I'm very glad to have the hibernate types package which helps me a lot accessing JSON(B) data type in my PostgreSQL database via Hibernate.

But currently I'm facing some performance issues when doing mass inserts for which I do some pre checks (looking up entites). That means, I do have arrays of numbers which I store as JSONB columns and as these numbers represent IDs I do check if those are valid IDs.

Actually my checkup code looks just about this:

Session session = HibernateUtil.getSession();
String queryString = "SELECT 1 as col from "+ tableName + " E where E.id= :id";
NativeQuery<?> query = session.createSQLQuery(queryString)
	.addScalar("col", new LongType());
query.setParameter("id", id);
//Integer result = (Integer) query.uniqueResult();
query.getSingleResult();

I'm using NativeQuery as I already tried to optimise performance through this, but actually it did not change anything from previous standard Hibernate query.

In the profiler >50% of the CPU time for the insert operation can be accounted for the checkups that perform code just like in my previous code snippet:

(the screenshot shows only one of multiple occurences in the call stack)

As you can see in the screenshot the 'performance bottleneck' is the calling of the method Class.getDeclaredMethod by ReflectionUtils.getDeclaredMethodOrNull. I'm a bit suprised as to why this is even called for my native sql query where I do only select constant '1' from a table.
That table on the other hand contains JSONB columns that are mapped in entities via the hibernate types package.

So my questions would be:

1. Is there anything I could change in my code to avert the calling of JsonTypeDescriptor.areEqual in such native queries?
2. Could there not possibly be done a performance optimisation within the hibernate types package in the way to not use reflection all the time and over and over again to "ask for something that was asked before"?

[mikoet]

For anyone interested in the bigger picture:

The underlined percentages of 41.6%, 14.6% an 17.5% will all result in calling of ReflectionUtils.getDeclaredMethodOrNull and Class.getDeclaredMethod in turn.

[vladmihalcea]

1. If you use a read-only Session, then flush is not called. Or, load the entities that triggered that dirty checking check in read-only mode.
2. Anything is possible. It's just Java code after all. Think of a good solution and share it with me. We could then discuss it and see if it can be integrated.

In your case, the problem comes from flushing, that's triggered by any SQL query execution.

If instead of an array, you use a Pojo that stores the array but implements equals and hashCode, those reflection calls could be prevented.

[vladmihalcea]

Closed as no further details were provided.

[masoncj]

Just as an aside, I'm also noticing this same hotspot in code that is very insert intensive.

[vladmihalcea]

You could avoid those problems like this:

If instead of an array, you use a Pojo that stores the array but implements equals and hashCode, those reflection calls could be prevented.

Try using a Pojo to represent your JSON objects.

[mikoet]

Hi,

I forgot to reply back then... I'm not sure if that's something one would put into such a library, but back then I made the following change which increased the performance quiet well:

index 86e301c..231a040 100644
--- a/hibernate-types-55/src/main/java/com/vladmihalcea/hibernate/type/json/internal/JsonTypeDescriptor.java
+++ b/hibernate-types-55/src/main/java/com/vladmihalcea/hibernate/type/json/internal/JsonTypeDescriptor.java
@@ -25,6 +25,7 @@ import java.lang.reflect.TypeVariable;
 import java.sql.Blob;
 import java.sql.SQLException;
 import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
 
 /**
  * @author Vlad Mihalcea
@@ -85,15 +86,33 @@ public class JsonTypeDescriptor
         if (one instanceof Collection && another instanceof Collection) {
             return Objects.equals(one, another);
         }
+        /*
         if (one.getClass().equals(another.getClass()) &&
             ReflectionUtils.getDeclaredMethodOrNull(one.getClass(), "equals", Object.class) != null) {
             return one.equals(another);
+        }
+         */
+        if (one.getClass().equals(another.getClass()) && getEqualsMethodOrNull(one.getClass()) != null) {
+            return one.equals(another);
         }
         return objectMapperWrapper.toJsonNode(objectMapperWrapper.toString(one)).equals(
             objectMapperWrapper.toJsonNode(objectMapperWrapper.toString(another))
         );
     }
 
+    private Map<Class<?>, Method> equalsMethodCache = new ConcurrentHashMap<>();
+    private Method getEqualsMethodOrNull(Class<?> clazz) {
+        Method method = equalsMethodCache.get(clazz);
+        if (method == null) {
+            method = ReflectionUtils.getDeclaredMethodOrNull(clazz, "equals", Object.class);
+            if (method != null) {
+                equalsMethodCache.put(clazz, method);
+            }
+        }
+        return method;
+    }
+
     @Override
     public String toString(Object value) {
         return objectMapperWrapper.toString(value);

Essentially I added a cache so that each method is only looked up once via reflection. I was not sure if concurrency was neccessary here and just didn't bother using a ConcurrentHashMap.

[vladmihalcea]

Thanks for the tip. I'll investigate it when I have some time.

[sverhagen]

I think I solved my performance problems with a mix of flush/clear and @Column(updatable = false), but I still wanted to add my two cents to the discussion, here.

Similar to if (one instanceof Collection && another instanceof Collection), I was wondering if there should be specific handling for Map in here, like so:

if (one instanceof Map && another instanceof Map) {
    return Objects.equals(one, another);
}

Map is not a Collection, and when it falls through to the objectMapperWrapper.toJsonNode ... equals approach, that is pretty expensive. (Admittedly, Map.equals isn't a huge improvement, but still better than having to JSON serialize.)

For context, I don't know why I was getting a LinkedHashMap here to begin with, but it's probably related to my use of generics in the entity:

public class JsonSourceEntity<T> extends BaseEntity {
    @Type(type = "jsonb")
    private T jsonData;
...

[vladmihalcea]

@sverhagen Please create a Pull Request with your change proposal and I'll review it.

[sverhagen]

Unfortunately, I can't commit to that at this time. As I said, I solved my performance problems in another way. Since you foreshadowed some further investigation in the future, I just wanted to leave some observations.

[vladmihalcea]

@sverhagen Fixed by #494