Module caching memory leak

[willsoto]

🐛 Bug Report

This is a crosspost of kulshekhar/ts-jest#1967 (Please read through this issue for context)

When transpiling TypeScript with a minimal transformer, there is a significant memory leak.

To Reproduce

Steps to reproduce the behavior:

1. git clone git@github.com:willsoto/jest-repro.git
2. git switch jest-memory-leak-custom-transpiler
3. npm ci
4. Run any of the test commands

Expected behavior

Modules should be cached once for the duration of the test run or allowed to be garbage collected after the test suite has completed.

Link to repl or repo (highly encouraged)

https://github.com/willsoto/jest-repro (switch to the jest-memory-leak-custom-transpiler branch)

envinfo

  System:
    OS: macOS 10.15.6
    CPU: (8) x64 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  Binaries:
    Node: 12.18.3 - ~/.volta/tools/image/node/12.18.3/bin/node
    Yarn: 1.22.4 - ~/.volta/tools/image/yarn/1.22.4/bin/yarn
    npm: 6.14.5 - ~/.volta/tools/image/npm/6.14.5/bin/npm
  npmPackages:
    jest: ^26.4.2 => 26.4.2

cc/ @ahnpnl

[ivanstanev]

Bump, we're still hitting this issue across several of our repos

[SethDavenport]

Do we know if this behaviour changed recently? Like did it used to not do this in an older version of Jest for example?

[bfaf]

I have exactly the same issue in my project. Fingers crossed to find a fix soon

Do we know if this behaviour changed recently? Like did it used to not do this in an older version of Jest for example?

I am observing this behavior with Jest 24.x onward
Memory leak is also observed in provided sample project. Just downgrade Jest version to 24.0.0 and run the tests

[abemedia]

Having the same issue when using 24.0.0 or anything above. Don't think it's related to the transformer though as I had the same issue when transpiling the TS and then running Jest on normal JS code.

Also see this repo: https://github.com/jaredjj3/jest-memory-leak-demo (however exposing GC did not fix it for me).

Exposing GC did reduce the leak by around 50%.

[jcbdev]

I put this commit on the ts-jest thread but it is relevant here i believe:

kulshekhar/ts-jest#1967 (comment)

When running with --expose-gc and --logHeapSize I was finding inconsistent results with sometimes the memory ballooning to over 6gb but others times never breaking 300mb.

The scenario I have found to make recreate it on my tests 100% of the time is to run jest with --no-cache or --clearCache before running the tests. If the cache is enabled then (obviously I guess) it can make this issue not apparent because loading the modules from the cache does not run the transformer and this does not have this leak

[takayukioda]

I'm having memory leak issue and it seems related.

Here's my reproducing sample.
https://github.com/takayukioda/reproduce-lab/tree/master/nest-e2e-memoryleak

here's commands to clone specific directory.

git clone --depth 1 --filter=blob:none --sparse https://github.com/takayukioda/reproduce-lab
cd reproduce-lab
git sparse-checkout init --cone
git sparse-checkout set nestjs-e2e-memoryleak

memory leak happens even if it doesn't use transformer.

[arekzaluski]

Any news about this problem. I'm stuck on jest and ts-jest 23 as 24+ or higher leads to memory leak. Calling GC manually doesn't fix it.
For my project:
Jest 23 and ts-jest 23: 1.2 GB heap size
Jest 24 and ts-jest 24 or higher: 12 GB heap size

[freezer278]

Any news? I have the same issue.

[juxtaposition]

Any news? I have the same issue.

Maybe you can try to isolate modules https://jestjs.io/docs/jest-object#jestisolatemodulesfn. my project is working with ts-jest and we are investigating if using the isolatedModules (sacrificing test checking) https://huafu.github.io/ts-jest/user/config/isolatedModules is the correct solution.

[avallete]

Maybe you can try to isolate modules https://jestjs.io/docs/jest-object#jestisolatemodulesfn. my project is working with ts-jest and we are investigating if using the isolatedModules (sacrificing test checking) https://huafu.github.io/ts-jest/user/config/isolatedModules is the correct solution.

Any luck with that mitigation ? We are currently hitting the same issues and wondering if it's worth a try.

[arikWaisman]

Any updates here?

[radcapitalist]

For my project, its more about version of Node.js than version of Jest. With Node 16.13, running our unit tests with --runInBand --logHeapUsage, only 20 test files complete before heap space runs out. Merely switching back to Node 14.15, all 81 of our test files complete. Heap usage really decelerates with Node 14.15. Heap usage reaches a higher number with Node 1415 (1818MB vs. 1472MB), but never hits out of memory. Graph of memory usage vs test file number:

Test files do not run in the exact same order, but close enough that that is not an issue.
Version of Jest for both tests was 27.4.4.

[SimenB]

Running with --detect-leaks keeps the usage stable, so I'll close this with the same reasoning as #7874 (comment)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.