Significant performance degradation after migration from 3.5.3 to 4.26.0

[Fedoop1]

Search first

• I searched and no similar issues were found

Description

After liquibase migration from 3.5.3 to 4.26.0 we noticed that the total liquibase update time became very unstable.
First of all overall memory consumption increased and now there are a few spikes that vary from 4 GB to 5 GB per execution (most of them are byte arrays) after which process private memory doesn't shrink. On 3.5.3 we didn't have such a problem.

The second problem is when there is memory pressure and ram utilization is close to the end, the total execution time can be several times longer than when there is enough memory for the execution. Before migration, it was stable for 3 minutes, but now it can last for 7-9 minutes. I'd guess that it's due to the problem mentioned above and probably OS can't dedicate so much memory to the liquibase process.

Steps To Reproduce

Please find the archive attached. You can run a test.ps1 script and monitor performance. There is only one big migration without sensitive data. NB: on the screenshot, you can see the performance overview from JMC of test db creation with the attached transaction included. Unfortunately, I can't attach all migrations due to their sensitivity.
liquibase-test.zip

Expected/Desired Behavior

Predictable average memory consumption. Shrink after spike. Predictable performance (execution time)

Liquibase Version

4.26.0

Database Vendor & Version

SQL Server 13.0.5893.48

Liquibase Integration

CLI

Liquibase Extensions

No response

OS and/or Infrastructure Type/Provider

Windows 10 Enterprise

Additional Context

I tried to look into this problem and found a few migrations after which I usually see some spikes. They are huge bulk inserts.
I moved SQL code from .xml changelog files to .sql and group insert statements into batches, spikes reduced from 4-5 GB to 2.5-3 GB per average execution, but it's still very high.

Sometimes I see strange runs when process memory doesn't exceed 1-2 GB and can't explain it to myself.

Are you willing to submit a PR?

• I'm willing to submit a PR (Thank you!)

[kevin-atx]

@Fedoop1, thank you for your thorough investigation and writeup of this issue. I will add this issue to the next sprint for review. Someone on our team will reach out if they have questions or need to discuss this further.