Circular references hang jest when assertions fail on node 14

[voces]

🐛 Bug Report

When an assertion fails where either the expected or actual value is circular, and both values are objects, jest encounters an error stating it failed to convert a circular structure to JSON, resulting in the test run not completing.

To Reproduce

it("test", () => {
  const foo = {};
  foo.ref = foo;

  expect(foo).toEqual({});
});

Running jest gives me the following error:

(node:11685) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    --- property 'ref' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (internal/child_process/serialization.js:117:20)
    at process.target._send (internal/child_process.js:804:17)
    at process.target.send (internal/child_process.js:702:19)
    at reportSuccess (/Users/verit/basic-jsx/node_modules/jest-worker/build/workers/processChild.js:67:11)

Jest continues running indefinitely (I only tested up to ten minutes) and reports nothing regarding the test suite.

I traced this to the added failureDetails property on error messages, landed in 26.3.0.

Expected behavior

I'd expect the test to fail and jest to complete running.

envinfo

I only tested two versions. The above error occurs on 14.9.0, but does not on 12.16.1.

  System:
    OS: macOS 10.15.6
    CPU: (8) x64 Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
  Binaries:
    Node: 14.9.0 - ~/.nodenv/versions/14.9.0/bin/node
    npm: 6.14.8 - ~/.nodenv/versions/14.9.0/bin/npm
  npmPackages:
    jest: ^26.4.2 => 26.4.2 

[Lonli-Lokli]

Is there any workaround?

[joelcoxokc]

@Lonli-Lokli Looks like the only workaround for now is --detectOpenHandles

However, this causes a massive decrease in performance.

I hope this can be fixed soon. I have migrated several of our projects to jest... and now it causes hiccups all throughout our build system when one test breaks.

It would be nice if --testTimeout worked in this scenario... but it still allows the test to just hang until Jenkins or circle ci times out.

[ziacik]

I can reproduce this when I have two such tests (in separate files - but not sure if this matters) and I have to run it with --watch. Here, however, it can be reproduced even without --watch: https://repl.it/@Frantiekiaik/jest-playground-1

[rimunroe]

I just ran into what I assume is the same issue on Node 10.16.0 and Jest 26.6.2. I can't reproduce it as written, but if I make a file containing two copies of @voces's example test and run it in watch mode, I get the same error.

The test:

it('test', () => {
  const foo = {};
  foo.ref = foo;

  expect(foo).toEqual({});
});
it('test 2', () => {
  const foo = {};
  foo.ref = foo;

  expect(foo).toEqual({});
});

The error:

(node:58951) ExperimentalWarning: The fs.promises API is experimental
(node:58951) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
    at JSON.stringify (<anonymous>)
    at process.target._send (internal/child_process.js:735:23)
    at process.target.send (internal/child_process.js:634:19)
    at reportSuccess (/Users/richardmunroe/analytics_ui/node_modules/jest-worker/build/workers/processChild.js:67:11)
(node:58951) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:58951) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Running with --detectOpenHandles makes the problem go away for me too.

[ziacik]

This is happening since #9496
Commit 9186361

And it seems to happen because of some inter-process serialization. After adding failureDetails property, the serialization fails on cyclic references as it contains the objects being under test. Maybe some sanitization of the failureDetails property would help.

[ziacik]

Note that after commit 5f6f2ec which changes default runner to circus, the error message is

"messageParent" can only be used inside a worker

      at messageParent (packages/jest-worker/build/workers/messageParent.js:46:11)

with JEST_JASMINE=1 the error is as before.

[ziacik]

Added a PR as an attempt to fix this.

Also, please note that the messageParent error mentioned above is due to this line which swallows the real error message which is also about circular references.

[SimenB]

Added a PR as an attempt to fix this.

🎉

Also, please note that the messageParent error mentioned above is due to this line which swallows the real error message which is also about circular references.

We should not swallow errors like that...

[piotrl]

We confirm it happens in Node 12, and it’s more common when using Angular Dependency Injection (I think they have cyclic structures in some error-states).

The process hangs in such scenario, but this can be improved slightly be applying --unhandled-rejection=strict to nodejs script, instead running jest as separate binary. It helps jest to recover and fail suite (but it does not resolve cyclic reference of course).

[daton89]

I can confirm this problem as very common with Angular DI with Node 12 as @piotrl mentioned.
Using Jest 26.6.3

How can I run it with --unhandled-rejection=strict flag?

    //package.json > scripts
    "test:unit": "node --unhandled-rejections=strict $(npm bin)/jest --env=jest-environment-jsdom-sixteen ",

this is actually stopping the execution, is that the right workaround?

[KrisBNelson]

I get this issue with mocking await functions in many tests in different describes in the same file which all fail when I have a mockReturnValueOnce in one describe, but doesn't appear with mockReturnValue in the describe block, it's as if the mocks are shared between describe blocks because the return value from an upper describe was being returned in my other describe block mock instead of its own return value...

node:internal/child_process/serialization:127
    const string = JSONStringify(message) + '\n';
                   ^

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property '_httpMessage' -> object with constructor 'Object'
    --- property 'socket' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:127:20)
    at process.target._send (node:internal/child_process:819:17)
    at process.target.send (node:internal/child_process:719:19)
    at reportSuccess (C:\node-projects\telus\tps\cdo-tps-quote-mgmt-api\node_modules\jest-worker\build\workers\processChild.js:59:11)

If I do jest --detectOpenHandles or jest --runInBand the errors don't appear, and I don't have a bunch of tests fail anymore.

[gaurav5430]

@KrisBNelson yeah, the circular reference error does not happen with jest --runInBand , why ?

[stephenh]

@gaurav5430 because with --runInBand everything (assertion failures/etc.) stays within a single Node process.

It's only when Jest has N child worker processes that, when a test fails in the child worker, the assertion failure needs to be serialized (via JSON) and sent to the parent process, and then a "cannot convert to JSON" error happens.

[gaurav5430]

@gaurav5430 because with --runInBand everything (assertion failures/etc.) stays within a single Node process.

It's only when Jest has N child worker processes that, when a test fails in the child worker, the assertion failure needs to be serialized (via JSON) and sent to the parent process, and then a "cannot convert to JSON" error happens.

what's the actual solution for this issue? would downgrading jest work ?

[simon-chen-1]

Any solution for this issue?

[G-Rath]

@SimenB gentle nudge to get your thoughts on what I've outlined in this and that comment :)

[yeahio]

@SimenB Kindly have this bug fixed, please!

[prapurnakumariTR]

hey, any solution on this issue ?

[theKashey]

Similar problem - expect(someDomeNode).toBe(someOtherNode) loops somewhere in the serialization of endless React/JSDOM structure and ends as an OOM exception.

[adamflush]

Node v16.19 Jest 29.5 simply calling a method that raises an error with a circular reference.

Try/catching the method call and replacing the error in the catch is a work-around. Without that, jest throws the unintuitive error message. Example:

       TypeError: Converting circular structure to JSON
        --> starting at object with constructor 'ClientRequest'
        |     property 'socket' -> object with constructor 'TLSSocket'
        --- property '_httpMessage' closes the circle
        at stringify (<anonymous>)

      at messageParent (node_modules/jest-worker/build/workers/messageParent.js:29:19)

The underlying dependency of the package throwing the error in this case is Axios.

The root problem with Jest is assuming JSON.stringify is always going to work. Use a safe stringify implementation, please.

[mrazauskas]

Currently a solution could be workerThreads: true option.

Note that it is marked experimental. That is recently added feature and it is not tested widely yet.

[obalilty]

i was getting the same error for ZoneJs object with circular structure
was able to workaround this issue by adding to following to testWorker.js (line 127)

const sendMessageToJest = (eventName, args) => {
  // start here
  if (args[1].failureDetails) {
    args[1].failureDetails.forEach((fd) => {
      fd.rejection = null;
      fd.promise = null;
      fd.zone = null;
      fd.task = null;
    })
  };
  // end here
  (0, _jestWorker().messageParent)([eventName, args]);
};

now getting the real error with stacktrace message and without any "Converting circular structure to JSON" errors
you may need to make adjustments depending on your code

A more correct approach might be to set null for each property of failureDetails item except 'stack' and 'message' props

jest v28.1.0 or 29.5.0
node v16.19.0

[tim-sh]

With jest@29.5.0, the following generic workaround helped me:

function deCircle(o, seen = new Set()) {
  if (!o || typeof o !== 'object') return;
  Object.entries(o).forEach(([k,v]) => {
    if (seen.has(v)) {
      o[k] = null;
      return;
    }
    const s = new Set(seen);
    s.add(v);
    deCircle(v, s);
  })
}
function reportSuccess(result) {
  if (!process || !process.send) {
    throw new Error('Child can only be used on a forked process');
  }
  deCircle(result); // ← patching the object
  process.send([_types.PARENT_MESSAGE_OK, result]);
}

You might need to patch on error as well, depending on the circumstances.

[felipebutcher]

I had this problem a few minutes ago. Found out it happened when I had two test files with same name in different folders. Once I renamed one of the files the problem was gone.

[danielo515]

Glad to see this is not a general problem with testing eslint rules. In my case, when a test fails, because almost all eslint nodes have circular references this problem is quite common.

[kasir-barati]

Still happening even though I am just expecting a number to be there and even thought it is but still tests fails 😭
Look here: https://github.com/kasir-barati/nestjs-materials/blob/main/typeorm/src/modules/talent-e2e/get-all.e2e-spec.ts#L9

The Solution that worked for me

#10577 (comment)

[kasir-barati]

Coming back again to the same issue, it seems that this time adding --detectOpenHandles flag is not gonna help me, I cannot deduce the issue from the logs and when I add that flag test suit fails immediately after it reaches the point that it should make a http req via axios (so no room to log anything in my test suit) and just logs that axios request failed with 400 http status code but in reality it even won't enter the catch block, any idea?

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'res' -> object with constructor 'Object'
    --- property 'req' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:159:20)
    at process.target._send (node:internal/child_process:852:17)
    at process.target.send (node:internal/child_process:752:19)
    at reportSuccess (/home/kasir/projects/you-say/node_modules/jest-worker/build/workers/processChild.js:82:11)

Node.js v20.10.0
 FAIL   backend-e2e  apps/backend-e2e/src/auth/auth-business.spec.ts
  ● Test suite failed to run

    Jest worker encountered 4 child process exceptions, exceeding retry limit

      at ChildProcessWorker.initialize (../../node_modules/jest-worker/build/workers/ChildProcessWorker.js:181:21)

[sawvox]

Coming back again to the same issue, it seems that this time adding --detectOpenHandles flag is not gonna help me, I cannot deduce the issue from the logs and when I add that flag test suit fails immediately after it reaches the point that it should make a http req via axios (so no room to log anything in my test suit) and just logs that axios request failed with 400 http status code but in reality it even won't enter the catch block, any idea?

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'res' -> object with constructor 'Object'
    --- property 'req' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:159:20)
    at process.target._send (node:internal/child_process:852:17)
    at process.target.send (node:internal/child_process:752:19)
    at reportSuccess (/home/kasir/projects/you-say/node_modules/jest-worker/build/workers/processChild.js:82:11)

Node.js v20.10.0
 FAIL   backend-e2e  apps/backend-e2e/src/auth/auth-business.spec.ts
  ● Test suite failed to run

    Jest worker encountered 4 child process exceptions, exceeding retry limit

      at ChildProcessWorker.initialize (../../node_modules/jest-worker/build/workers/ChildProcessWorker.js:181:21)

Trying to stringify the req object from Axios will result in the circular reference: req -> res -> req
As your error states:

--> starting at object with constructor 'Object'
| property 'res' -> object with constructor 'Object'
--- property 'req' closes the circle

Try comparing the properties of the request or response object that you care about rather than the whole object itself.

I.e. expect(res.body?.myProperty).toEqual("foo")

[kasir-barati]

So @sawvox what you're saying is that if I just compare what I wanted in the request or response object I would not face this issue but I guess that is not something to compromise, But thanks for uncovering one more mystery for me. I did not know that it is happening because of my expects.

[franzos]

Today I've started seeing similar errors on a repository that I've been working on for 4 years. Funny thing is, it's really random - sometimes it works, sometimes it doesn't. Flags like --runInBand seem the increase the likelihood that it works, but it's no guarantee.

node:internal/child_process/serialization:159
    const string = JSONStringify(message) + '\n';
                   ^

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'socket' -> object with constructor 'Object'
    --- property '_httpMessage' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:159:20)
    at process.target._send (node:internal/child_process:838:17)
    at process.target.send (node:internal/child_process:738:19)
    at reportSuccess (/usr/src/app/node_modules/.pnpm/jest-worker@29.7.0/node_modules/jest-worker/build/workers/processChild.js:82:11)

Node.js v18.16.1

Also tried the suggested workerThreads: true but this will lead to a different error:

    DataCloneError: function transformRequest(data, headers) {
        const contentType = headers.getContentType() || '';
        const ha...<omitted>... } could not be cloned.

      at messageParent (../../node_modules/.pnpm/jest-worker@29.7.0/node_modules/jest-worker/build/workers/messageParent.js:24:34)

This is on Jest 29.5.12.