Memory Leak on ridiculously simple repo #7874

javinor · 2019-02-12T17:00:20Z

You guys do an awesome job and we all appreciate it! 🎉

🐛 Bug Report

On a work project we discovered a memory leak choking our CI machines. Going down the rabbit hole, I was able to recreate the memory leak using Jest alone.

Running many test files causes a memory leak. I created a stupid simple repo with only Jest installed and 40 tautological test files.

I tried a number of solutions from #7311 but to no avail. I couldn't find any solutions in the other memory related issues, and this seems like the most trivial repro I could find.

Workaround :'(

We run tests with --expose-gc flag and adding this to each test file:

afterAll(() => {
  global.gc && global.gc()
})

To Reproduce

Steps to reproduce the behavior:

git clone git@github.com:javinor/jest-memory-leak.git
cd jest-memory-leak
npm i
npm t

Expected behavior

Each test file should take the same amount of memory (give or take)

Link to repl or repo (highly encouraged)

https://github.com/javinor/jest-memory-leak

Run `npx envinfo --preset jest`

Paste the results here:

System:
    OS: macOS High Sierra 10.13.6
    CPU: (4) x64 Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
  Binaries:
    Node: 10.15.0 - ~/.nvm/versions/node/v10.15.0/bin/node
    Yarn: 1.12.3 - /usr/local/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v10.15.0/bin/npm
  npmPackages:
    jest: ^24.1.0 => 24.1.0

The text was updated successfully, but these errors were encountered:

milesj · 2019-02-12T18:00:53Z

I found this out recently but you can use the Chrome console to debug Node scripts! You can try using the Chrome console to profile Jest while it's running to try and dig into the issue.

I believe the command is: node --inspect ./node_modules/.bin/jest --watch -i. When running, open Chrome and go to about:inspect. You should then see the running Node script.

jeysal · 2019-02-12T20:21:40Z

Do I understand correctly that using the workaround to force GC runs makes the heap size remain constant? In that case it's not really a memory leak, just v8 deciding not to run the GC because there is enough memory available. If I try running the repro with 50MB heap size

node --max_old_space_size=50 node_modules/.bin/jest --logHeapUsage --runInBand --config=jest.config.js

the tests still complete successfully, supporting this assumption.

javinor · 2019-02-12T21:26:11Z

@milesj I ran through some memory dumps, but couldn't make much sense of them, I'm not too experienced with pursuing leaks and I didn't want to point in the wrong direction without something solid to count on.

@jeysal you are right of course! The thing is our tests freeze in the middle of running since (I assume and could be wrong) we run out of memory. After spending a lot of time trying to figure this out, I found #7274. It seemed to me from the discussion that the behaviour I encountered here is not intended. wdyt @SimenB ?

javinor · 2019-02-22T18:26:17Z

Bueller?

alfaproject · 2019-03-21T22:34:08Z

My tests are also leaking massively on CI but the exact same setup locally doesn't really leak (much at least).

It's so bad, I'm considering disabling tests on CI until I can make sense of what the difference is beside the OS. ):

javinor · 2019-03-25T09:24:45Z

Hey guys!

I simplified the memory leak case to a single file which runs tautological tests and eventually throws an exception due to a memory leak.

I'm not sure how to move forward with this... help? @SimenB @jeysal @milesj

jest_leak@1.0.0 test /Users/javinor/workspace/jest-memory-leak
> jest --logHeapUsage --runInBand --config=jest.config.js


 RUNS  __test__/repeat100.test.js

<--- Last few GCs --->

[82597:0x103800000]    79124 ms: Mark-sweep 1259.1 (1442.2) -> 1240.0 (1436.2) MB, 438.3 / 0.0 ms  (average mu = 0.310, current mu = 0.304) allocation failure scavenge might not succeed
[82597:0x103800000]    79197 ms: Scavenge 1255.7 (1436.2) -> 1242.6 (1438.2) MB, 15.9 / 0.0 ms  (average mu = 0.310, current mu = 0.304) allocation failure
[82597:0x103800000]    79256 ms: Scavenge 1256.5 (1438.2) -> 1244.7 (1441.7) MB, 11.0 / 0.0 ms  (average mu = 0.310, current mu = 0.304) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2440d84dbe3d]
Security context: 0x30802859e6e9 <JSObject>
    1: prepareStackTrace [0x3080c9459f79] [/Users/javinor/workspace/garbage/jest_leak/node_modules/source-map-support/source-map-support.js:~388] [pc=0x2440d854b545](this=0x308094d07b59 <JSFunction Error (sfi = 0x308018616e01)>,error=0x3080e18623f9 <Error map = 0x3080f3c619f9>,stack=0x3080e1862e91 <JSArray[23]>)
    2: InternalFrame [pc: 0x2440d848ee75]
    ...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003c597 node::Abort() [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 2: 0x10003c7a1 node::OnFatalError(char const*, char const*) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 3: 0x1001ad575 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 4: 0x100579242 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 5: 0x10057bd15 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 6: 0x100577bbf v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 7: 0x100575d94 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 8: 0x10058262c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 9: 0x1005826af v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
10: 0x100551ff4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
11: 0x1007da044 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
12: 0x2440d84dbe3d
Abort trap: 6

fyodorvi · 2019-03-27T19:33:09Z

Similar here, jest + ts-jest, simple tests get over 1GB of memory and eventually crash.

kirillgroshkov · 2019-03-28T12:33:56Z

crashes for us too

jeysal · 2019-03-30T13:00:49Z

@javinor For a test file containing a ridiculous number of tests, I'm not sure there's much we can do, we have to keep the test objects around until the test file is finished - this is the heap while the tests are running:

Hundreds of MBs of test objects, so for the case shown in this repro we can't really help either :/

thymikee · 2019-03-30T13:18:38Z

FYI @scotthovestadt is currently working on holistically improving memory efficiency of Jest, so improvements are coming (some of them in the next minor version).

kirillgroshkov · 2019-03-30T15:06:16Z

I wonder, why isn't it possible for Jest to spawn a process for each test file, which will guarantee that memory will be freed? Ok, it can be slower, of course, but in my case - it's much better to be slower rather than get a crash from out-of-memory and be blocked to use Jest alltogether...

Maybe an option? Or a separate "runner" (not sure if I understand architecture and terminology right)?

Is it architecturally possible?

Or, will Node-experimental-workers solve it?..

scotthovestadt · 2019-03-30T17:18:17Z

I've made a few improvements to memory in the next release:
#8234
#8233

I have a future plan to improve memory in a couple of ways:

by not holding onto test results at all, which are currently held until the end. related: Support JSON line output. #8242
limiting number of modules that can exist in the require cache

The problem with your suggestion of just spawning a new worker for each test is that it would be very slow. A better suggestion along the same lines would be to monitor the memory usage of the processes and auto-restart them at some threshold. I have some concerns about that in general, I'd rather always fix memory leaks than paper them over, but if a PR did that I would accept it.

Let me know if the release next week helps with the problems you've been experiencing.

kirillgroshkov · 2019-03-30T21:38:12Z

@scotthovestadt thanks for the info! I'll definitely check with the next release.

My actual issue is reported here: #8247
And there I'm dealing with much bigger leaks, +30Mb per each test file (~150 files in my project).

javinor · 2019-04-01T07:40:43Z

Thanks for the responses guys!

I think I can break this down to different two problems:

Memory consumption grows the more test files I have
Memory consumption grows the more tests I have per file

We're running thousands of tests, each creating a relatively big setup so we get bitten twice.

The original screenshot showing the consumption growing from test file to test file, hinting to a leak between tests - I have a few guesses as to why this happens, but nothing solid yet.

The exception I referred to later, as far as I can tell, really has to do with what @jeysal pointed out - having a large number of tests in the file. In our case, we have only hundreds of tests but with a very large setup. I'll try to provide a better reproduction of this.

I'll update after the next release, when I get to poke around a bit more and see the additional fixes in action. Thanks guys!

alfaproject · 2019-04-01T10:25:50Z

There must be something else wrong because I'm currently using Jest v23.6 and everything works fine, no memory leaks, no anything.

If I upgrade to latest Jest then the memory leaks start to happen, but only on the GiLab CI runner. Works fine locally.

SimenB · 2019-04-02T10:53:03Z

New release is out: https://github.com/facebook/jest/releases/tag/v24.6.0

alfaproject · 2019-04-02T14:52:30Z

Meh, it's still leaking in my setup ):

�[0KRunning with gitlab-runner 11.8.0 (4745a6f3)
�[0;m�[0K  on the-mill-ec2-gitlab-runner GvW8cLGR
�[0;m�[0KUsing Docker executor with image cypress/base:8 ...
�[0;m�[0KUsing locally found image version due to if-not-present pull policy
�[0;m�[0KUsing docker image sha256:2ad7a04a3a1d41fac2585d88e2dbd19ed3df9c57df2b27276c30e4f317c2c8e3 for cypress/base:8 ...
�[0;msection_start:1554216455:prepare_script
�[0KRunning on runner-GvW8cLGR-project-10463077-concurrent-0 via runner-gvw8clgr-gitlab-runner-1554212599-0c63c2c8...
section_end:1554216456:prepare_script
�[0Ksection_start:1554216456:get_sources
�[0K�[32;1mCloning repository...�[0;m
Cloning into '/builds/the-mill/lambda-gaming'...
�[32;1mChecking out a592f4d2 as update-fe-bo-packages...�[0;m
�[32;1mSkipping Git submodules setup�[0;m
section_end:1554216459:get_sources
�[0Ksection_start:1554216459:restore_cache
�[0Ksection_end:1554216460:restore_cache
�[0Ksection_start:1554216460:download_artifacts
�[0K�[32;1mDownloading artifacts for nodeModulesFe (189107055)...�[0;m
Downloading artifacts from coordinator... ok      �[0;m  id�[0;m=189107055 responseStatus�[0;m=200 OK token�[0;m=iQz-3io7
section_end:1554216481:download_artifacts
�[0Ksection_start:1554216481:build_script
�[0K�[32;1m$ cd ./packages/frontends�[0;m
�[32;1m$ node --version�[0;m
v8.12.0
�[32;1m$ node --expose-gc ./node_modules/.bin/ng test --ci --code-coverage=false --run-in-band�[0;m
PASS apps/whitelabel/src/app/app.component.spec.ts (195 MB heap size)
  AppComponent
    ✓ should create the app (87ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        5.168s
Ran all test suites.
PASS apps/backoffice/src/app/layout/layout.component.spec.ts (294 MB heap size)
PASS apps/backoffice/src/app/app.component.spec.ts (289 MB heap size)

Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   1 passed, 1 total
Time:        4.154s
Ran all test suites.
PASS libs/api/src/lib/api.module.spec.ts (364 MB heap size)
  ApiModule
    ✓ should create (69ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        3.283s
Ran all test suites.
PASS libs/core/src/lib/user.service.spec.ts (427 MB heap size)
PASS libs/core/src/lib/auth.service.spec.ts (426 MB heap size)
PASS libs/core/src/lib/wallet.service.spec.ts (426 MB heap size)
PASS libs/core/src/lib/unauth.guard.spec.ts (428 MB heap size)
PASS libs/core/src/lib/auth.guard.spec.ts (428 MB heap size)
PASS libs/core/src/lib/route.service.spec.ts (426 MB heap size)
PASS libs/core/src/lib/core.module.spec.ts (426 MB heap size)

Test Suites: 7 passed, 7 total
Tests:       8 passed, 8 total
Snapshots:   0 total
Time:        6.945s
Ran all test suites.
PASS libs/ui/src/lib/form-errors/form-errors.component.spec.ts (507 MB heap size)
PASS libs/ui/src/lib/loader/loader.component.spec.ts (507 MB heap size)
PASS libs/ui/src/lib/css-url.pipe.spec.ts (506 MB heap size)
PASS libs/ui/src/lib/ui.module.spec.ts (526 MB heap size)

Test Suites: 4 passed, 4 total
Tests:       6 passed, 6 total
Snapshots:   2 passed, 2 total
Time:        6.525s
Ran all test suites.
PASS libs/registration/src/lib/registration-form/registration-form.component.spec.ts (5.068s, 633 MB heap size)
PASS libs/registration/src/lib/registration.module.spec.ts (628 MB heap size)

Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   1 passed, 1 total
Time:        7.861s
Ran all test suites.
PASS libs/casino-game/src/lib/casino-game/casino-game.component.spec.ts (687 MB heap size)
PASS libs/casino-game/src/lib/casino-game.module.spec.ts (685 MB heap size)

Test Suites: 2 passed, 2 total
Tests:       4 passed, 4 total
Snapshots:   1 passed, 1 total
Time:        3.839s
Ran all test suites.
PASS libs/documents/src/lib/document-upload-form/document-upload-form.component.spec.ts (736 MB heap size)
PASS libs/documents/src/lib/documents/documents.component.spec.ts (736 MB heap size)
PASS libs/documents/src/lib/documents.module.spec.ts (736 MB heap size)

Test Suites: 3 passed, 3 total
Tests:       5 passed, 5 total
Snapshots:   2 passed, 2 total
Time:        4.32s
Ran all test suites.
PASS libs/forgot-password/src/lib/forgot-password-form/forgot-password-form.component.spec.ts (818 MB heap size)
PASS libs/forgot-password/src/lib/forgot-password/forgot-password.component.spec.ts (818 MB heap size)
PASS libs/forgot-password/src/lib/forgot-password.module.spec.ts (837 MB heap size)

Test Suites: 3 passed, 3 total
Tests:       5 passed, 5 total
Snapshots:   2 passed, 2 total
Time:        7.06s
Ran all test suites.
PASS libs/reset-password/src/lib/reset-password-form/reset-password-form.component.spec.ts (922 MB heap size)
PASS libs/reset-password/src/lib/reset-password/reset-password.component.spec.ts (919 MB heap size)
PASS libs/reset-password/src/lib/reset-password.module.spec.ts (929 MB heap size)

Test Suites: 3 passed, 3 total
Tests:       5 passed, 5 total
Snapshots:   2 passed, 2 total
Time:        7.922s
Ran all test suites.
PASS libs/payment/src/lib/payment-process/payment-process.component.spec.ts (1018 MB heap size)
PASS libs/payment/src/lib/payment-option-list/payment-option-list.component.spec.ts (1016 MB heap size)
PASS libs/payment/src/lib/payment-option/payment-option.component.spec.ts (1011 MB heap size)
PASS libs/payment/src/lib/card-form/card-form.component.spec.ts (1013 MB heap size)
PASS libs/payment/src/lib/payment.service.spec.ts (1013 MB heap size)
PASS libs/payment/src/lib/payment.module.spec.ts (1032 MB heap size)
PASS libs/payment/src/lib/card-type-detector.directive.spec.ts (1023 MB heap size)

Test Suites: 7 passed, 7 total
Tests:       11 passed, 11 total
Snapshots:   4 passed, 4 total
Time:        14.741s
Ran all test suites.
PASS libs/fluid-container/src/lib/fluid-container/fluid-container.component.spec.ts (1081 MB heap size)
PASS libs/fluid-container/src/lib/fluid-container.module.spec.ts (1081 MB heap size)

Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   1 passed, 1 total
Time:        4.164s
Ran all test suites.
PASS libs/layout/src/lib/layout/layout.component.spec.ts (6.025s, 1181 MB heap size)
FAIL libs/layout/src/lib/sidebar/sidebar.component.spec.ts
  ● Test suite failed to run

    TypeScript diagnostics (customize using `[jest-config].globals.ts-jest.diagnostics` option):
    �[96mlibs/layout/src/lib/sidebar/sidebar.component.spec.ts�[0m:�[93m26�[0m:�[93m7�[0m - �[91merror�[0m�[90m TS2739: �[0mType '{ __typename: "NavList"; classes: string[]; navListItems: undefined[]; }' is missing the following properties from type 'NavList': horizontal, id

    �[7m26�[0m       navList: {
    �[7m  �[0m �[91m      ~~~~~~~�[0m

      �[96mlibs/layout/src/lib/sidebar/sidebar.graphql.ts�[0m:�[93m8�[0m:�[93m3�[0m
        �[7m8�[0m   navList: NavList;
        �[7m �[0m �[96m  ~~~~~~~�[0m
        The expected type comes from property 'navList' which is declared here on type 'Sidebar'

PASS libs/layout/src/lib/header/header.component.spec.ts (1213 MB heap size)
PASS libs/layout/src/lib/footer/footer.component.spec.ts (1194 MB heap size)
PASS libs/layout/src/lib/layout.module.spec.ts (5.055s, 1212 MB heap size)

Test Suites: 1 failed, 4 passed, 5 total
Tests:       7 passed, 7 total
Snapshots:   3 passed, 3 total
Time:        19.157s
Ran all test suites.
PASS libs/cms/src/lib/page/page.component.spec.ts (7.531s, 1269 MB heap size)
PASS libs/cms/src/lib/dialog-route/dialog-route.component.spec.ts (1271 MB heap size)
PASS libs/cms/src/lib/content-group/content-group.component.spec.ts (1272 MB heap size)
PASS libs/cms/src/lib/cms.service.spec.ts (1270 MB heap size)
PASS libs/cms/src/lib/slot.directive.spec.ts (1268 MB heap size)
PASS libs/cms/src/lib/cms.module.spec.ts (1270 MB heap size)

Test Suites: 6 passed, 6 total
Tests:       9 passed, 9 total
Snapshots:   3 passed, 3 total
Time:        18.367s
Ran all test suites.
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[10:0x2510510]   136242 ms: Mark-sweep 1362.8 (1460.0) -> 1362.8 (1460.0) MB, 1827.9 / 0.1 ms  allocation failure GC in old space requested
[10:0x2510510]   137760 ms: Mark-sweep 1362.8 (1460.0) -> 1362.8 (1429.0) MB, 1518.2 / 0.1 ms  last resort GC in old space requested
[10:0x2510510]   139285 ms: Mark-sweep 1362.8 (1429.0) -> 1362.8 (1429.0) MB, 1524.6 / 0.1 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xb1cf5fa5879 <JSObject>
    1: stringSlice(aka stringSlice) [buffer.js:560] [bytecode=0x1e161b35dd29 offset=94](this=0x11a4d26822d1 <undefined>,buf=0x20928062d71 <Uint8Array map = 0x35a39b242941>,encoding=0xb1cf5fb5409 <String[4]: utf8>,start=0,end=1097377)
    2: toString [buffer.js:633] [bytecode=0x1e161b35d979 offset=145](this=0x20928062d71 <Uint8Array map = 0x35a39b242941>,encoding=0xb1cf5fb5409 <String[4]: utf8>,st...

 1: node::Abort() [ng test --ci --code-coverage=false --run-in-band]
 2: 0x8cbf4c [ng test --ci --code-coverage=false --run-in-band]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [ng test --ci --code-coverage=false --run-in-band]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [ng test --ci --code-coverage=false --run-in-band]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [ng test --ci --code-coverage=false --run-in-band]
 6: v8::internal::Factory::NewStringFromUtf8(v8::internal::Vector<char const>, v8::internal::PretenureFlag) [ng test --ci --code-coverage=false --run-in-band]
 7: v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [ng test --ci --code-coverage=false --run-in-band]
 8: node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding, v8::Local<v8::Value>*) [ng test --ci --code-coverage=false --run-in-band]
 9: 0x8e4a76 [ng test --ci --code-coverage=false --run-in-band]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [ng test --ci --code-coverage=false --run-in-band]
11: 0xb0bbec [ng test --ci --code-coverage=false --run-in-band]
12: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [ng test --ci --code-coverage=false --run-in-band]
13: 0x220bb08042fd
/bin/bash: line 90:    10 Aborted                 (core dumped) node --expose-gc ./node_modules/.bin/ng test --ci --code-coverage=false --run-in-band
section_end:1554216631:build_script
�[0Ksection_start:1554216631:after_script
�[0Ksection_end:1554216633:after_script
�[0Ksection_start:1554216633:upload_artifacts_on_failure
�[0K�[32;1mUploading artifacts...�[0;m
�[0;33mWARNING: ./packages/frontends/junit/*.xml: no matching files�[0;m 
�[31;1mERROR: No files to upload                         �[0;m 
section_end:1554216634:upload_artifacts_on_failure
�[0K�[31;1mERROR: Job failed: exit code 1
�[0;m

peterdemartini · 2019-04-05T19:36:59Z

After updating to 24.6.0, we are seeing the similar issue running our CI tests. When logging the heap usage, we see an increase of memory usage after each test file.

scotthovestadt · 2019-04-07T19:40:11Z

This should help: #8282

Will be released soon.

alfaproject · 2019-04-24T16:31:14Z

How soon? )':

Supernats · 2019-05-23T17:22:55Z

For those reading along at home, this went out in 24.8.0.

unional · 2019-07-18T18:49:57Z

The problem with your suggestion of just spawning a new worker for each test is that it would be very slow.

This would also be a huge breaking change.

unional · 2019-07-18T18:53:16Z

If I upgrade to latest Jest then the memory leaks start to happen, but only on the GiLab CI runner. Works fine locally.

Also encounter out of memory issue with jest + ts-jest:
https://circleci.com/gh/unional/standard-log/382

Originally I thought it was some circular dependency on my source code, but may be jest related.
So throwing this into the mix. 🌷

Supernats · 2019-07-18T18:56:34Z

@unional if you're on Circle, make sure maxWorkers isn't higher than the CPU allotted you by Circle.

EDIT: To be clear, you should proactively specify maxWorkers at or below the CPU allotted you Circle.

unional · 2019-07-18T19:01:58Z

@Supernats thanks. I think I did have that set during the failure, currently I'm running it with --runInBand 😢

But it still fail once in a while:
https://circleci.com/gh/unional/standard-log/448

ulrichb · 2019-07-29T16:23:49Z

I have Jest 24.8.0 and #8282 doesn't seem to help. Also --runInBand only helps a bit (4 GB instead of 10 GB 😮).

Pleaaaaaaase fix this ...

kirillgroshkov · 2019-07-29T16:26:14Z

Yes, following this thread for long since it still fails for us and in ~10% of the cases runs with "out of memory" for CircleCI 2Gb RAM instances.

mbyrne00 · 2022-06-27T23:44:10Z

And we're already starting to run into problems with libs requiring a specific min LTS version of node now (as per my linked issue above). This is getting painful to work around without dodgy --ignore-engines for CI.

V8 issue seems to be closed as WontFix so got no idea what the longer term solution is: https://bugs.chromium.org/p/v8/issues/detail?id=12198

Link to node issue nodejs/node#40014

mbyrne00 · 2022-06-28T00:56:22Z

It seems there is a suggested fix/workaround for Jest as per this comment: https://bugs.chromium.org/p/v8/issues/detail?id=12198#c20

Hopefully this makes more sense to someone in the Jest team ... is this something that could be persued? It seems the first suggestion is for node itself but for jest they are asking if it's possible to remove forced GCs. I gotta admit I don't know the detail.

Did Victor's suggested workaround work for Node? Updating from above, it would be to change https://source.chromium.org/chromium/chromium/src/+/main:v8/src/heap/heap.h;l=1460;drc=de8943e4326892f4e584a938b31dab0f14c39980;bpv=1;bpt=1 to remove the is_current_gc_forced_ check.

In general it's my understanding that --exposed-gc is primarily a testing feature and shouldn't be depended upon in production. Is it not possible to remove forced GCs from how jest runs?

emilioSp · 2022-07-20T13:40:09Z

Same issue here.
For me, it seems that the problem is in the setupFilesAfterEnv script.

It's better with node 16.10, but it still arrives at 841 MB heap size (580 tests)

pedroeckel · 2022-07-27T19:57:44Z

Same issue here too. Downgrading to node 16.10 fixed the memory issue with Jest.

mrkutly · 2022-08-04T16:38:34Z

Same issue for me. Downgrading to node 16.10 fixed the memory leak with Jest. I was seeing 3.4GB heap sizes with node 16.14, down to ~400MB with node 16.10.

ChandlerFerry · 2022-08-05T04:07:23Z

Looking for some forward momentum here as well; CI pipeline is locked at 16.10 and explicitly calls node --expose-gc ./node_modules/.bin/jest to reduce heap sizes from ~3699MB to 610MB on two repositories.

It hasn't resolved the issue for all repositories, so I suspect this issue more complex.

Previous chromium issue linked by @mbyrne00 is labeled as wont-fix, which is concerning to me with Node 18 becoming LTS in October.

Is there any solution in mind?

mbyrne00 · 2022-08-06T00:36:12Z

The chromium issue points back to node and jest and how they are implementing certain features. As one user on the Chromium issue thread pointed out, this is the closest thing to an attempt to resolve the issue so hopefully it gets legs: https://github.com/facebook/jest/pull/12205/files#diff-c0d5b59e96fdc7ffc98405e8afb46d525505bc7b1c24916b5c8482de5a186c00R1359-R1373

Even more interesting is this specific comment where someone has published a jest runtime lib to fix the problem until the PR is merged #12205 (comment)

StringEpsilon · 2022-08-06T02:34:16Z

Just FYI, the interesting news on the node / v8 upstream bug can be found at #11956 ("Memory consumption issues on Node JS 16.11.0+") - where some mitigations for the upcoming jest 29 and a workaround (albeit a very slow one) for jest 27 were discussed just 4 days ago.

As far as I understood @SimenB's intention with reopening this issue, it is NOT to discuss known causes of memory leaks, but to track the overall state of jest leakages outside of identified and known issues and causes.

That is to say, if your leak goes away when downgrading to node 16.10, this is not the issue for you. Go to #11956 instead. :)

chrisandrewcl · 2022-10-30T14:32:49Z

I've run some tests considering various configurations. Hope it helps someone.

node version	node args	jest args	custom behavior	time (seconds)	heap (mb)
16.10	--expose-gc --no-compilation-cache	--maxWorkers 1	afterAll(global.gc) + force options.serial to false on jest-runner	303	45
16.18	--expose-gc --no-compilation-cache	--maxWorkers 1	afterAll(global.gc) + force options.serial to false on jest-runner	325	47
16.10	--expose-gc --no-compilation-cache	--maxWorkers 2	-	236	64
16.18	--expose-gc --no-compilation-cache	--maxWorkers 2	-	167	67
16.10	--expose-gc	--maxWorkers 1	afterAll(global.gc) + force options.serial to false on jest-runner	234	82
16.10	--expose-gc	--maxWorkers 2	-	155	96
16.10	--expose-gc --no-compilation-cache	--runInBand --detectLeaks	afterAll(global.gc)	313	159
16.10	--expose-gc --no-compilation-cache	--runInBand --detectLeaks	-	307	160
16.10	--expose-gc --no-compilation-cache	--runInBand	-	313	160
16.10	--expose-gc --no-compilation-cache	--maxWorkers 1	-	333	160
16.10	--expose-gc --no-compilation-cache	--runInBand --detectLeaks	afterEach(global.gc)	397	160
16.18	--expose-gc --no-compilation-cache	--runInBand --detectLeaks	afterAll(global.gc)	281	164
16.18	--expose-gc --no-compilation-cache	--runInBand --detectLeaks	afterEach(global.gc)	298	164
16.18	--expose-gc --no-compilation-cache	--maxWorkers 1	-	287	165
16.18	--expose-gc --no-compilation-cache	--runInBand --detectLeaks	-	300	165
16.18	--expose-gc --no-compilation-cache	--runInBand	-	337	165
16.10	--expose-gc	--runInBand --detectLeaks	-	258	199
16.10	--expose-gc	--runInBand	-	247	201
16.10	--expose-gc	--maxWorkers 2	-	286	201
16.10	--expose-gc	--runInBand --detectLeaks	afterAll(global.gc)	256	202
16.10	--expose-gc	--runInBand --detectLeaks	afterEach(global.gc)	309	206
16.10		--runInBand	-	261	629
16.18	--expose-gc	--maxWorkers 2	-	277	899
16.18	--no-compilation-cache	--runInBand	-	297	907
16.18		--runInBand	-	281	1055
16.18	--expose-gc	--runInBand	-	347	1262
16.18	--expose-gc	--maxWorkers 1	afterAll(global.gc) + force options.serial to false on jest-runner	337	1380

Test Suites: 3 skipped, 31 passed, 31 of 34 total
Tests:       20 skipped, 49 todo, 171 passed, 240 total
Snapshots:   0 total

* Running with Jest 29.2.2 on a bitbucket pipeline container using node official docker images

Versions of Axios 1.x are causing issues with Jest (axios/axios#5101). Jest 28 and 29, where this issue is resolved, has other issues surrounding memory leaks (jestjs/jest#7874). Allow `>=0.25.0` for applications that cannot upgrade Jest at this moment.

wyardley · 2023-03-08T00:58:25Z

From a preliminary run or two, it looks to me like going back to 16.10 is resolving these errors for us as well.
Is there any more clarity on why this is, or what a real fix might look like?

StringEpsilon · 2023-03-08T06:42:51Z

Is there any more clarity on why this is, or what a real fix might look like?

All the info on the regression that specifically affects node >= 16.11 is found in this issue: #11956

earloc · 2024-02-09T08:31:19Z

Just spend about 2 days figuring out how to overcome this, until I discovered #11956.

TLDR; regression introduced in node 16.11, fixed in 21.1.
Backporting Bugfix for node 18 seems to be on the way, see
#11956 (comment)

MattLishmanYara · 2024-02-09T12:07:05Z

Just spend about 2 days figuring out how to overcome this, until I discovered #11956.

TLDR; regression introduced in node 16.11, fixed in 21.1. Backporting Bugfix for node 18 seems to be on the way, see #11956 (comment)

In case anyone stumbles across this and wants a simple solution, node 20.10.0 contains a fix for this.
Upgrading to node 20.10.0 solved this for me 🎉

earloc · 2024-02-09T21:04:26Z

Reading the linked issue, it says '21.1', but might as well being already backported to '20.x', leaving '18.x' to be waiting for a fix🤷.

For our team, switching our ci-builds to '21.x' did the deal, even if this might introduce runtime-confusion 😉.

javinor added the 🐛 Bug label Feb 12, 2019

merlinnot mentioned this issue Oct 1, 2019

Add an option to allow module caching #8993

Closed

sr258 mentioned this issue Nov 9, 2019

Test cases run out of memory Lumieducation/H5P-Nodejs-library#221

Closed

mbyrne00 mentioned this issue Jun 27, 2022

Support node 16.10 auth0/node-oauth2-jwt-bearer#65

Closed

apoorv-mishra mentioned this issue Aug 9, 2022

perf: reduce memory usage upon running server tests outline/outline#3949

Merged

Tirokk mentioned this issue Aug 17, 2022

🐛 [Bug] FATAL ERROR In Backend Tests, Because We Run Out Of Memory Ocelot-Social-Community/Ocelot-Social#5180

Open

3 tasks

alessbell mentioned this issue Sep 23, 2022

Explore @swc/jest as an alternative to ts-jest apollographql/apollo-client#10121

Closed

thewatermethod mentioned this issue Sep 29, 2022

[TTAHUB-1037] Identify and fix memory leak issue HHS/Head-Start-TTADP#1050

Merged

10 tasks

vojto mentioned this issue Oct 11, 2022

Memory leak when running multiple tests firebase/firebase-js-sdk#6664

Closed

joachimvh mentioned this issue Nov 2, 2022

Look into jest memory issues CommunitySolidServer/CommunitySolidServer#1496

Closed

christhompsongoogle mentioned this issue Dec 30, 2022

Unit test memory leak firebase/firebase-tools#4836

Closed

daneryl mentioned this issue Jan 9, 2023

ci api unit test run out of memory half of the runs, leaving the test running forever. huridocs/uwazi#5418

Closed

SevInf mentioned this issue Jan 9, 2023

Possible memory leak when running tests with Jest prisma/prisma#17171

Closed

SevInf mentioned this issue Feb 14, 2023

Prisma Memory Leak when using in NestJS app, reported by Jest prisma/prisma#17925

Closed

mike12345567 mentioned this issue Mar 27, 2023

Fix for user groups adding users to app user metadata tables Budibase/budibase#10120

Merged

sheremet-va mentioned this issue Apr 18, 2023

feat!: support running tests using VM context vitest-dev/vitest#3203

Merged

racs4 mentioned this issue Jul 13, 2023

How to clear used memory between test suites? (possible memory leak) nodkz/mongodb-memory-server#776

Closed

TheSisb mentioned this issue Jan 26, 2024

fix(summary-detail): fixes issue in react <18 twilio-labs/paste#3741

Merged

Memory Leak on ridiculously simple repo #7874

Memory Leak on ridiculously simple repo #7874

Comments

javinor commented Feb 12, 2019

🐛 Bug Report

Workaround :'(

To Reproduce

Expected behavior

Link to repl or repo (highly encouraged)

Run npx envinfo --preset jest

milesj commented Feb 12, 2019

jeysal commented Feb 12, 2019

javinor commented Feb 12, 2019

javinor commented Feb 22, 2019

alfaproject commented Mar 21, 2019

javinor commented Mar 25, 2019

fyodorvi commented Mar 27, 2019

kirillgroshkov commented Mar 28, 2019

jeysal commented Mar 30, 2019

thymikee commented Mar 30, 2019

kirillgroshkov commented Mar 30, 2019

scotthovestadt commented Mar 30, 2019

kirillgroshkov commented Mar 30, 2019

javinor commented Apr 1, 2019 • edited

alfaproject commented Apr 1, 2019

SimenB commented Apr 2, 2019

alfaproject commented Apr 2, 2019

peterdemartini commented Apr 5, 2019 • edited

scotthovestadt commented Apr 7, 2019

alfaproject commented Apr 24, 2019

Supernats commented May 23, 2019

unional commented Jul 18, 2019

unional commented Jul 18, 2019

Supernats commented Jul 18, 2019 • edited

unional commented Jul 18, 2019 • edited

ulrichb commented Jul 29, 2019

kirillgroshkov commented Jul 29, 2019

mbyrne00 commented Jun 27, 2022 • edited

mbyrne00 commented Jun 28, 2022 • edited

emilioSp commented Jul 20, 2022

pedroeckel commented Jul 27, 2022

mrkutly commented Aug 4, 2022 • edited

ChandlerFerry commented Aug 5, 2022 • edited

mbyrne00 commented Aug 6, 2022 • edited

StringEpsilon commented Aug 6, 2022 • edited

chrisandrewcl commented Oct 30, 2022

wyardley commented Mar 8, 2023

StringEpsilon commented Mar 8, 2023

earloc commented Feb 9, 2024

MattLishmanYara commented Feb 9, 2024

earloc commented Feb 9, 2024

Run `npx envinfo --preset jest`

javinor commented Apr 1, 2019 •

edited

peterdemartini commented Apr 5, 2019 •

edited

Supernats commented Jul 18, 2019 •

edited

unional commented Jul 18, 2019 •

edited

mbyrne00 commented Jun 27, 2022 •

edited

mbyrne00 commented Jun 28, 2022 •

edited

mrkutly commented Aug 4, 2022 •

edited

ChandlerFerry commented Aug 5, 2022 •

edited

mbyrne00 commented Aug 6, 2022 •

edited

StringEpsilon commented Aug 6, 2022 •

edited