Memory consumption for generated SqlObject DAO

[alexcase52]

Hi, I'm using JDBI 3.43 and I'm using generated Dao classes. The memory consumption per call looks to be too big. The issue seems to be depending on the number of methods in Dao.

sample test

    @Test
    public void get() throws Exception {
        for (int i = 0; i < 1000; i++) {
            TestDao dao = rig.jdbi.onDemand(TestDao.class);
            var courses = dao.getCoursesByName("some");
        }
    }

Dao looks like the one below. For the test I added just 20 similar methods.

@GenerateSqlObject
public interface TestDao {
    @SqlQuery("SELECT * FROM course WHERE name=:newCourseName")
    @UseRowMapper(TestDao.CourseMapper.class)
    List<CoursePO> getCoursesByName(@Bind("newCourseName") String newCourseName);

   .....
    @SqlQuery("SELECT * FROM course WHERE name=:newCourseName")
    @UseRowMapper(TestDao.CourseMapper.class)
    List<CoursePO> getCoursesByName19(@Bind("newCourseName") String newCourseName);
}

Jdbi is configured like that:

                    result = Jdbi.create(ds);
                    result.installPlugin(new PostgresPlugin());
                    result.installPlugin(new SqlObjectPlugin());
                    result.installPlugin(new VavrPlugin());
                    result.registerArgument(new PGobjectArgumentFactory());
                    result.registerArgument(new PageFileNameArgumentFactory());
                    result.registerArgument(new RightsArgument.Factory());
                    result.registerColumnMapper(new RightsArgument.Mapper());
                    result.registerColumnMapper(new TimeMappers.LocalDateColumn());
                    result.registerColumnMapper(new TimeMappers.LocalDateTimeColumn());
                    result.registerColumnMapper(new TimeMappers.UtilDateColumn());

The result in profiler is following:

It looks like generated implementation does very heavy initialization for all methods in class, even if just one method is needed in fact.

[stevenschlansker]

Thanks for reporting this. Yes, it's expected that Jdbi will eagerly initialize all methods for a class. This assures that if there is any serious misconfiguration, it is reported up-front, rather than failing later. That said, this does look like a fair amount of memory usage. Your screenshot doesn't quite show the whole picture: which objects actually end up holding all that memory? but we will try to look into this.

[alexcase52]

Hi @stevenschlansker,
Thank you for the quick reply!

The problem is that for each sequential Dao call, all this ConfigRegistries initialization will happen again. So moving TestDao testDao = rig.jdbi.onDemand(TestDao.class); out of cycle doesn't improve the situation. It results in heavy memory allocation/garbage collection.

I think, being more specific on what configs are prepared/warmed up (e.g. for called method only) or somehow caching this config in Dao object for all subsequent calls will let solve this. (Or even generation for a call fluent API-like code... )

Using JDBI with generated SqlObject is very eager on memory but good for db access code management, fluent API is much more lightweight but efficient.

Memory allocation for the same 1000 calls with fluent API

10 times cheaper! 8)

[alexcase52]

I looked for some workarounds. I created default method in the interface and implemented the call myself as

@GenerateSqlObject
interface TestDao extends SqlObject {
    default long someSelect() {
     try (var handle = getHandle()) {
                return handle.select("""
                                 < some query >
                                """, args)
                        .mapTo(Long.class)
                        .one();
            }
    }
}

The optimization didn't work because in generated class I still have the code as:

class TestDaoImpl {
...
 @Override
    public long someSelect() {
      return (long) jdbi.withExtension(TestDao.class, e -> ((TestDaoImpl) e).someSelect());
    }
}

My guess is that for default methods the generated code may call default method directly.

[stevenschlansker]

Hi, I'm not sure you'll have too much luck finding workarounds - but we should be able to improve memory usage with some code improvements. I do not think we can make your suggested change to call default methods directly, at least not in all cases, because default methods may have decorating annotations like @Transaction that need additional logic.