Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(jest-runner): switch workers to advanced serialization #14030

Closed
wants to merge 2 commits into from

Conversation

alex3d
Copy link

@alex3d alex3d commented Mar 27, 2023

Summary

PR would switch workers from current JSON serialization to advanced (structuredClone) serialization.
This would fix problems with BigInt(#11617), circular references (#10577) and probably some other problems.

Test plan

Enable already existing BigInt test in e2e/__tests__/complexItemsInErrors.ts and test in e2e/__tests__/circularInequality.test.ts

@facebook-github-bot
Copy link
Contributor

Hi @alex3d!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@alex3d
Copy link
Author

alex3d commented Mar 27, 2023

cc @SimenB
It seems you have had plans to switch to advanced serialization after Node 12 EOL: #11467 (comment)

I've added basic worker result sanitization to pass https://github.com/facebook/jest/blob/b38d7d345a81d97d1dc3b68b8458b1837fbf19be/e2e/__tests__/complexItemsInErrors.ts#L31
Probably this should be removed along with test to make child_process workers more in line with new experimental worker_threads runner.

@facebook-github-bot
Copy link
Contributor

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

Copy link
Contributor

@mrazauskas mrazauskas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

It is worth to consider landing the sanitizeTestResult() function separately, because it is fixing one of the bugs I have hit while using workerThreads worker.

@@ -85,14 +85,31 @@ const sendMessageToJest: TestFileEvent = (eventName, args) => {
messageParent([eventName, args]);
};

const sanitizeTestResult = (result: TestResult) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better add it as a function:

Suggested change
const sanitizeTestResult = (result: TestResult) => {
function sanitizeTestResult(result: TestResult) {

@mrazauskas
Copy link
Contributor

By the way, I was testing this branch locally. In the Node docs it is mentioned that structure clone algorithm:

supports more built-in JavaScript object types, such as BigInt, Map and Set, ArrayBuffer and TypedArray, Buffer, Error, RegExp etc.

However, this format is not a full superset of JSON, and e.g. properties set on objects of such built-in types will not be passed on through the serialization step.

How to understand this? Does it mean that extra properties set on an Error object can lead to some bugs in assertion error messages? I was trying to figure this out, but everything worked smooth.

@alex3d
Copy link
Author

alex3d commented Mar 28, 2023

Yeah, node docs are right: extra properties of Error are stripped after structuredClone. But result.testResults[0].failureDetails[0] actually is a plain object. I'm not familiar enough with jest internals to tell why it is not an Error.

image

image

It is worth to consider landing the sanitizeTestResult() function separately, because it is fixing one of the bugs I have hit while using workerThreads worker.

Should I open new PR?
Was that workerThreads bug something like expect(someFunction).eq(someOtherFunction)?
Also, I was thinking about more JSON.stringify like approach of deleting function properties from TestResult recursively, but jest test suite didn't show up any cases when function properties could appear in other places.

@mrazauskas
Copy link
Contributor

Should I open new PR? Was that workerThreads bug something like expect(someFunction).eq(someOtherFunction)?

The output in console was similar to "handles functions that close over outside variables" test. It was appearing while I was refactoring code base with huge objects, but leaving some mistakes in tests. In a way tests were supposed to fail, only the message was odd. Since object were huge, I did not manage to reproduce the error so far.

Not sure about separate PR. Let’s wait for maintainer’s feedback.

@github-actions
Copy link

This PR is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Jun 26, 2023
@github-actions
Copy link

This PR was closed because it has been stalled for 30 days with no activity. Please open a new PR if the issue is still relevant, linking to this one.

@github-actions github-actions bot closed this Jul 26, 2023
@github-actions
Copy link

This PR was closed because it has been stalled for 30 days with no activity. Please open a new PR if the issue is still relevant, linking to this one.

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants