Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(serialize): remove surrial #2877

Merged
merged 12 commits into from May 7, 2021
Merged

feat(serialize): remove surrial #2877

merged 12 commits into from May 7, 2021

Conversation

nicojs
Copy link
Member

@nicojs nicojs commented May 5, 2021

Replace surrial with JSON.stringify and JSON.parse to serialize and deserialize the messages between Checker and TestRunner worker processes.

This has a couple of advantages:
✅ Improves performance (slightly).
✅ Removes a (small) security risk where an eval~ish way was used to deserialize.
✅ Reduces weird behavior in edge cases where you provide functions in Stryker options.
✅ Reduces maintenance load by removing a fringe dependency (that I maintain, but still 😅).

However, this does mean that an undocumented feature disappears (a small, but breaking change).

// stryker.conf.js
module.exports = { 
  karma: {
    webpack: {
      some: /regex/ // 👈 this will be send as `null` to the worker process.
    }
  }
};

This is why Stryker will provide an optional warning:

image

BREAKING CHANGE: Having a non-JSON-serializable value in your configuration won't be sent to the child process anymore. If you really need them in your test runner configuration, you should isolate those values and put them in test runner-specific config files, loaded by the test runner plugin itself, for example, jest.config.js, karma.conf.js, webpack.config.js.

Closes #2814

@bartekleon
Copy link
Member

bartekleon commented May 6, 2021

I did forget to push my changes to github 😅. This actually looks almost the same as what I have done, so I believe it is good. Since 2 people independently have done the same work

@nicojs
Copy link
Member Author

nicojs commented May 6, 2021

Ah sorry, I wanted to move forward with this as it is one of the last things that needs to be done before we can release v5. 🤷‍♂️

I'm busy with a small piece of code to verify that your stryker config is fully serializable. If not, I want to give a warning. Something like:

[WARN] Configuration option "path.to.config[0].option" isn't serializable. If a test runner or checker plugin is dependent on this value, it might not receive it as intended. Reason: Primitive type "function" has no JSON representation.

This is what I currently have:

https://www.typescriptlang.org/play?target=7&ssl=1&ssc=1&pln=69&pc=1#code/MYGwhgzhAEAKCmAnCB7AdtA3gKAL7eHVRHgDoQUBzACgDMBLNAEwFU0Il6wR6AvMAEYkI1TNGzRJ0NAC5oAZgA0EqQHpV0AfUqMALnIAsAJjTKp0CAE8AtgLkBlGwJQhqASjNTas6LQCuaMC69OjumLiekmCIiHIA2gC6kdAoAgBWcjjm5rrwEPpZ2UUADkZyaPAA7nBIqGjuKkX45hGNmigucrqIfvDJ+bHQAORDyQFMcuPwDBVMyYz5YIHw5VXQoJAQ4Q24bm4A3NhHeki0YMDw0GwciFw8-ELwACJ5wLfFwehYbcVgugAWcgGjEoiUOOX+IMmaAA1mgUJU0OCpIh4JB0EDuiDDvhsP5Ap8MExpox4EwADwAFQAfNQAVDoJToAAfaBTGZklnSPwgEBuLqQtCUaD0GBMwqSVG6PyIDD0oXQACEAF5lWzmCTZtAAGTa6Dy4UqtVoHkgHFHfFBEIYGasdicbh8QTCOmCyjQuEItD8q7226Oh4kF4QN70D7WxJc9mkpjfcwQSr0XTAf7QOmWYrwFC0fVutxxorrSCXIYm2xIIYyNrmYDF4bAoWV6tSWscYbOFxotBNwtFttDaOzHu9lHwaWy5FFVslrQ6NC6YdTutDS2ExfZaf1pwude9qUyjBxCUj8y-AHxJLNk8G5InyVoupyAAGsFu1iT9AAbpddBnLgAiAASTBf0zbNcxBXB-2gf5IGkFBoAAKXsAB5AA5aBUWKVEOHnP5rSfK9slwBJJw3ZdUjSeAgl3Qt1GgABRJN-iQaAljgFDkJQxRuV5HjokQMBLBSRAUnSajdBFdhdCWC4iPMegc1dEFoFVY1TXzY870wscDzI3tmjvRS0wAQRiITSFFMzBMsZShT2AttMkQhpIgoUYDVA1SFocBdAAWTAYpqGoFN6BAOYpOJAAPfNlWpXxGDtG47idR4RFC8K3AAflIaxAuoYkQ1EuK00Kt5SDPf5SACCBIVoXRqES+AotIXQUHsLEhXcDxoDKxA9jcbywtyRACs1MkDnkwt91lNzKAgch4CFAFoCyuaYGhYkOSYfTC0Mk8XPyObXxQNrVOgFDxKCUhKDHE62tA+AUNoOzKEm7TjNe+6ELUi6rt0CrEFO06-xZVkDW+1TVV4vlHKcw7JINDy-qo66lqxPJXsGny-gCoLqDiGF4EsHjP24BJYvi21rgde5nUxsm+RyvKgr61T4rGoqKr+KqarqhqicsHq+oGoaQBGsbtvepz73HOU3QWkhltTNakegTbxp2qbJH2kcEegbDgYAYXAKA0LAaxLk8t1SARnoghQRBSDQC3LmZVkhnJJZ0BsFA-BgDYoGpIZdqnIhO3IKhqCGQ22pNzZzct0YDaBuPTYgRO+hT4308zxblZ42OUHjs3XahtURjcbWdLl6Aj2r08eYvW87xvBvR3RHwX1TkHMxFGAgMwIuS4z12oJ4+kYGKFxfcQYpIWAXqxzAMLoETXloFuySKCOpgZRU5KA3w9BSEI7SSLI-B8CAA

@bartekleon
Copy link
Member

sth like that?:
https://www.typescriptlang.org/play?target=7#code/MYGwhgzhAEAKCmAnCB7AdtA3gXwFC+HVRHgDoQUBzACgDMBLNAEwFU0Il6wR6AvMAEYkI1TNFzRoaAFzQAzABoJ0AQBdZAFgBMaJZIgBPALYDZAZWMCUIagEo90WjMcBXNMFX10dnA7CJEWQBtAF0HFAEAK1lMZUlVeAh1WMlU6AAHLVk0eAB3OCRUNDs46DxJbAcra1lVRBd4ByTA6AByVoc3Jlku+AYcpgdGJLB3eGy86FBICBwS7FtbAG58XEYExFowYHhoNg5ELh5+IXgAEUTgQ-TPdCxldLBVAAtZZsZKUJXJRHhIdDedQ+KzwuFobg8XgwTD6jHgTAAPAAVAB81BeH1kSOgAB9oL1+vDcVIXCAQLZas8PtB6DBsSloL9VC5EBgMWhKNAAIQAXh5+OYsIG0AAZCLoOzObz+WhSSAQfhwe5bhh+qx2JxuHxBMJ0VSOT00ABrNAoXJoCl7DWHLUnEgXCBXeg3KGhYkEuFMe76XL0VTAZ7QdEGdLwFC0CX6yi2b2pYCQXatWUmJCtaSleMcNrvDlpjMJtrVEijPNpTOJj0DUtpRnwZms75xgutAT0SjrauScttJWQ9CdqbNwwmawDn51lkYIIMmsZJ6vaChByz37-ZwAA1ghyMfvoADddqoQ7sAEQAEkwR9D4cjH2wJ+gz0gUhQ0AAUmYAPIAOVr6V+HBoKoTxQuupQVCEjZds2ESRPAHhjtAAD0SHQAAon6zxINAoxwJ+H6fgoJJkkR-iIGABjQCgiBUVE8GqDS7DAWM4E0hGerUnyMpyjGM41kyk5Qak5Q1vQ7EAIIBBRpC0pJ5EGBxHKLLGs6EExt4cjA-KSqQtDgKoACyYDpNQ1ABvQICDIxMIAB4xjyKKOIw6oHEc2qnCI5mWbYAD8pBGMZ1Awo6NEOUGwVXKQjwvKQbgQFStCqNQznwDZpCqCgZhAhydj2NAEWIIsti6RZGxBUK8LLKx471myUYQOQ8Aci80A+RplAwIaMKEkwQkVKxalJO1W4oBl0D8p+dEeKQlB1iNGVXvAn60Ip0Z9WxQaSvNr5cdAk1wdN-6jaNx64niW2IMd418sR5IqTWg0MZKWl7VNqikE1QKJKtxV6U8RkmdQQRGvABhEXu3AhPZjlqvsmrHDq30Q+SfkBSZBXjY55UhVF86xewCVJSDBh5QVRUlSAZXdZ6VWzjVk7tQ1JDNYGbXPdAXUVb1rEiWWRAMUdGUAMLgFA35gEYuzaVGpCPfUHjUaQaAS7sOJ4q0CKjOgxgoC4MDTFAKKtOtg3WGQFA0K0gsoCLMzi5LHQZJdwuixA9uNE7x222LKuNSzRHW97bsq9d-LtLYrECayi58Wk0ULku1W1musibs7J2hjSMDnpggeu+795ERiMDpNYOuIOkVLAPldZgBZ0C+mS0CzQxFBDUwLLUq5togegpBgbO2CQcoeCgkAA

@bartekleon
Copy link
Member

working on printing messages

@bartekleon
Copy link
Member

bartekleon commented May 6, 2021

@nicojs good?

https://www.typescriptlang.org/play?target=7#code/MYGwhgzhAEAKCmAnCB7AdtA3gXwFC4Acxl4AReCYRASwIBdr0AKAM2rQBMBVNCJasCGoAvMACMQFJpmi5o0NAC5oAZgA0c6GLrKALACY0G+RACeAWzHKAyhbEoQTAJTHoLJW4CuaYA2ZOcV2JEZQBtAEY1aH0olm9fRjRnLGwAXVcUMQArZUxNeToKHTz5UugCfWU0eAB3OCRUJKd86Dx5bFd7B2U6RE94Vwhe5QByEddvDmVJ+DZqjld2IbAfeCra6FBICBxnXGwnJwBufFx2QsQWMGB4aB4+GkERcUlyShp6RKxNIjoAC2UQxoaAA5qFUid5Ih4JB0IDeuwQSc8Lg4j4-BgOLN2PAOAAeAAqAD4mP9EcoCdAAD7QGZzXHUhSeEAgJw9P6I6DUGCUkrQaF0TyIDBk0HQACEAF5JbTONj5tAAGSK6CikES6VMlnI-BohLoNzsbi8fhPUQSKRq6ZoADWaBQNTQbLuJseQnNrwoVFoGPBjLpOI43xMNWodGAf2gpNMBHgKBYqo5oKcwdKwEgtxGaE8liQI0ULXTfGgIyBiPzhYzJa6khWFbKRczAfm9bK-PgguFkLTVZGYmoIPOrfkjZLeoxw82vbMlgck6hHaFGFCfLb5TA-zC6RaZWhsI8AANYDRzGHqAA3W50GO3ABEABJMNfY-HE4jsLfoH9IAoUNAAFLWAA8gAcu2BDQnwaB0BuiQHjurQQpWxYjJkWTwL487QAA9Nh0AAKJhn8SDQCscBAYBQFRNmLJRMEYCmNAKCIEx2QYXQXK8DBqwIdQCakkm6rSjKNGsqma4Cku3ZlG0bZ8VGACCiCIAxAB03JKSppgCYihziW2wDoEMb6gjAMpqqpLDgHQACyYAEEwTARtQIALJxWIAB4ppKRKGpw9ymu6LxSM5rlOAA-Kp5j2UwWLvNAPlRnFVCqb8fyqd4EAciwdBMEa8AeapdAoNYCKgs4LjQMliCHE4lkuRcsXyrixwIQunYioJECqZIoL-NA4UmSCMDWli9IcNJpSyQ2RkcWqx4oMVCXQEBbG+KpIIdgtxXPvAQEsDpyaTfI8mHSC21-sJK1rXQqWIIti03tSNLzfdS1XaJKargZs1DWZ13oet8DQTQlqCXVVkbnZDlMKENrwKYUTnoIqTeb5czGg8AhBRaEBMMjrKRdFDnVQlvlNe8qUbulmXZbl8OmJV1W1fVICNWNgatWu7VLn9PXAyC-WDWqI2yhz8zHa0CGGVx5RvSgADC4BQCBYDmLc5mCapMtAp4vjMapaBq7cVI0iMeIrOgFgoJ4MBbFARIjJLOsOPAPUoCCTAjBBD1K9sqvq+Mcu+8rEABwMwfFX7KvG-zfV-FEPtR6H4cJZqYzNBJi7CtAK5teum659u3PtvuyhHvLu1cjAD6YEnisp8bH5RGSMAEA41uIAQHLAFVHZgC50Chiy0CbRxIAoMZHBCpyWNmrB6CqfBa5pN2eAouOXxECQsDU0waXwsCYKo6mkgcUQy23mlRUoNr6BsCCt7diwzFOb9BBMQmaVfS0fF5WHYAQL72IHwAAktBfetV9LyAvgAahlAeVSj4CDYAPJNbA0B4AgGLN9GBYBoDwOgAeUIyC0hoJaLJWSkkc5EB1KieIGJ1wkDeN6T4zBqo+kSMoAKbpngWhYR8X0qR-RynGj-E6B0OFsIwFdZsLV9LULQGvTQL9EBMDPlyZaAAGI4mi8R93eJwxevVBZ-F0dQWBsDxFpiMq7d2nsELL25qEAA6gpAASiBYRCt74DiFAvDAKBpHQFrtvPgu9-gU1Yb6agqQqb-CcB+auaARgcTnjjSQqloCgITPgwoxk+hoGqCxZimxiLAHhixAgIBPCDgwNyPusY5TQSYp1BpBN+hRDDNAU8II-gcXtBxaENwLy3G6T+c4wMsQcCye4mEjRlDHmoKeBgl5VRPUfFImJcS9yNGmqURx+AyiZyliiIAA

@nicojs
Copy link
Member Author

nicojs commented May 6, 2021

Ow sorry, I was thinking just use paths.join('.') (basic json path). We shouldn't spent too much time on it 🤷‍♂️

@bartekleon
Copy link
Member

bartekleon commented May 6, 2021 via email

@nicojs nicojs marked this pull request as ready for review May 6, 2021 20:14
@nicojs
Copy link
Member Author

nicojs commented May 6, 2021

I think this is ready for review. @kmdrGroch would you mind taking a look? 👀

@@ -7,7 +7,7 @@
"checkCoverage": true,
"reporter": "html",
"reportDir": "reports/coverage",
"exclude": [
"testResources/**/*.js"
"include": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happened there? Exclude no longer needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we should have used an include all along. I noticed it because of this failing pipeline. Coverage was calculated on the unit test files themselves, instead of the source files 😅

We're also using an include in all of our .nycrc.json files, don't know why not here.

@@ -221,6 +221,11 @@
"description": "decide whether or not to log warnings when a preprocessor error occurs. For example, when the disabling of type errors fails.",
"type": "boolean",
"default": true
},
"unserializableOptions": {
"description": "decide whether or not to log warnings when a configuration options is unserializable. For example, using a /regex/ in your config options.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe "configuration options are unserializable"

@@ -127,7 +127,7 @@ export class ChildProcessProxy<T> implements Disposable {

private listenForMessages() {
this.worker.on('message', (serializedMessage: string) => {
const message: ParentMessage = deserialize(serializedMessage, [File]);
const message: ParentMessage = deserialize(serializedMessage);
Copy link
Member

@bartekleon bartekleon May 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially could use

const message = deserialize<ParentMessage>(serializedMessage);

to keep it consistent as it is in other files
plus that's why we have

function deserialize<T>(s: string): T

return options;
}

function markUnknownOptions(options: StrykerOptions, schema: JSONSchema7, log: Logger): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is explicit return type needed here? We are not exporting that function

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it isn't. But it is allowed, isn't it? I'll remove these explicit return types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicit return types are bugprone. So if we don't have to, better let's not do it :D

}
}

function markUnserializableOptions(options: StrykerOptions, log: Logger): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. Explicit return type is no needed IMO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

"@types/lodash.flatmap": "~4.5.6",
"@types/node": "^14.0.1"
"@types/node": "^14.0.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all around me are familiar faces, trailing spaces, trailing spaaceesss

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😆. removing...

export function findUnserializables(thing: unknown): UnserializableDescription[] | undefined {
switch (typeof thing) {
case 'number':
return isNaN(thing) || thing === Infinity || thing === -Infinity
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch! Instead of checking +- Infinity you can do !isFinite(thing)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, great catch. Didn't even know about isFinite. It also catches NaN apparently, so replaced all 3 checks with 1 isFinite check 👍 https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/isFinite#return_value

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow... i knew about isFinite but didn't know it catches NaN as well :D
we all learn hahaha

@@ -91,13 +91,14 @@
"source.fixAll.eslint": true
},
"cSpell.words": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you even using cSpell.words? 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using it. But indeed, we should add it to our recommended plugins. It is awesome! Been adding words in our config (using ctrl+. and choosing "add this word to workspace").

image

@bartekleon
Copy link
Member

In total looks good. Only small details IMO :)

@nicojs nicojs enabled auto-merge (squash) May 7, 2021 09:57
@nicojs nicojs merged commit 5114835 into epic/v5 May 7, 2021
@nicojs nicojs deleted the feat/remove-surrial branch May 7, 2021 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants