Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large XML Reports cause crash due to OOM #192

Open
benjaminstokes opened this issue Dec 16, 2021 · 0 comments
Open

Large XML Reports cause crash due to OOM #192

benjaminstokes opened this issue Dec 16, 2021 · 0 comments

Comments

@benjaminstokes
Copy link

Background

When fetching a report that is large (e.g. 500MB+) CxFlow can throw out of memory exceptions and crash. This is more common in memory constrained environments such as Github Actions where 7GB of memory is available for the task, even when that available memory is allocated for JVM heap space via JVM flags.

Suggested Solution
The reports need to be processed in a more memory efficient manner to prevent these crashes so that large reports will not crash CxFlow.

Details

Based on several stack traces, the issue occurs within getReportContent here: https://github.com/checkmarx-ltd/checkmarx-spring-boot-java-sdk/blob/develop/src/main/java/com/checkmarx/sdk/service/CxService.java#L381

The XML report is treated as a String type and several operations are performed on the string creating several pieces of garbage for the GC. When the report is large, these pieces of garbage are equally large and will quickly consume all available heap space.

The XML report is read into memory as a string:
https://github.com/checkmarx-ltd/checkmarx-spring-boot-java-sdk/blob/develop/src/main/java/com/checkmarx/sdk/service/CxService.java#L395

This line creates two pieces of garbage by trimming the string twice. This commonly triggers the OOM error.

https://github.com/checkmarx-ltd/checkmarx-spring-boot-java-sdk/blob/develop/src/main/java/com/checkmarx/sdk/service/CxService.java#L400

In other log examples, this function also causes OOM:
https://github.com/checkmarx-ltd/checkmarx-spring-boot-java-sdk/blob/develop/src/main/java/com/checkmarx/sdk/service/CxService.java#L402

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant