Implement mutation switching #2340
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Temporarily disable e2e tests while we work in the mutation switching epic.
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 2.33.0 to 2.34.0. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v2.34.0/packages/eslint-plugin) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Add parsers to the instrumenter package. Add support for: * Parsing `*.js`, `*.tsx`, `*.html` and `*.vue` files. * Load custom babel config with `babel.config` format and `.babelrc` format.
Add printers for html, ts, vue and js files.
Add mutant placers. * `switchCaseMutantPlacer`: places a mutant in a switch case * `conditionalExpressionMutantPacer`: places a mutant in a conditional expression
Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse) from 7.9.6 to 7.10.1. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/master/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.10.1/packages/babel-traverse) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
…#2218) Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 2.33.0 to 2.34.0. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v2.34.0/packages/parser) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Bumps [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) from 7.9.6 to 7.10.1. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/master/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.10.1/packages/babel-core) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.9.6 to 7.10.2. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/master/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.10.2/packages/babel-preset-env) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Bumps [@babel/cli](https://github.com/babel/babel/tree/HEAD/packages/babel-cli) from 7.8.4 to 7.10.1. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/master/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.10.1/packages/babel-cli) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Add transformers. Transformers are responsible for the process of generating mutants and placing them in the AST. * `htmlTransformer`: Capable of transforming html AST * `babelTransformer`: Capable of transforming babel AST (JS and TS)
Bumps [flatted](https://github.com/WebReflection/flatted) from 2.0.2 to 3.0.2. - [Release notes](https://github.com/WebReflection/flatted/releases) - [Commits](WebReflection/flatted@v2.0.2...v3.0.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Bumps [@babel/parser](https://github.com/babel/babel/tree/HEAD/packages/babel-parser) from 7.9.6 to 7.10.2. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/master/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.10.2/packages/babel-parser) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Bumps [@babel/types](https://github.com/babel/babel/tree/HEAD/packages/babel-types) from 7.9.6 to 7.10.2. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/master/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.10.2/packages/babel-types) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Implement `Instrumenter` class, responsible for: * Parsing code to an AST * Mutant generation based on AST * Transforming the code to incorporate the mutants in mutation switches * Printing AST back to code
Bumps [@types/mkdirp](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/mkdirp) from 1.0.0 to 1.0.1. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/mkdirp) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Adds `Checker` plugin type, to be used in #1514 (and can later be used to filter mutants in #1980). The idea is that, before mutation testing (or even before files are instrumented), we initialize the checker. It can be used as a sanity check. The `TypescriptChecker` can use this to load all files and compile once (with `--no-emit`) to make sure errors don't occur under normal circumstances. The `check` method is called one-by-one on each mutant. A checker can decide to assign any of a `CheckStatus`. If it `Passed` it proceeds to the next checker, or if it was the last checker it will be run in a test runner. If another status is assigned, it will stop processing that mutant and report it as done.
Removes some calls to deprecated Jasmine APIs.
| Old | New |
|--|--|
| `throwOnExpectationFailure` | `jasmine.Env.configure( { oneFailurePerSpec: true } )` |
| `stopOnSpecFailure` | `jasmine.Env.configure( { failFast: true } )` |
| `jasmine.addReporter` | `jasmine.env.addReporter` |
| `jasmine.clearReporters` | `jasmine.env.clearReporters` |
Add `id` property to `Mutant` api
Update instrumenter syntax with latest design of test runner api. * `__mutationCoverage__` -> `__mutantCoverage__` * `activeMutant` --> `__activeMutant__`
Adds a new test runner API to Stryker. Differences with the old API: 1. Explicit difference between a `dryRun` (initial test run) and a `mutantRun` 1. A mutantRun result has an `activeMutant` field indicating which mutant should be activated by the runner 2. The result types are also explicit. A `MutantRunResult` now reports back the status as `Killed` or `Survived` instead of letting Stryker make that decision. 2. Code coverage interfaces are replaced with the simpler `MutantCoverage` interface. 3. The `MutantRunOptions` now has a `testFilter`, which replaces the old `TestFramework` API. Add small improvements * Add an `id` field to a `TestResult` and using that id for the filtering instead of the cumbersome `TestSelection` interface we introduced to handle string identifiers (names) * `TestResult`, `DryRunResult` and `MutantRunResult` to be implemented using [discrimitated unions](https://www.typescriptlang.org/docs/handbook/advanced-types.html#discriminated-unions). This removes a lot of the optional fields. For example, `errorMessage` is now a required field on an `ErrorDryRunResult`. * Defined `errorMessage` and `failureMessage` to be a `string`, instead of a `string[]`. If runners want to report more messages, they will have to concatenate themselves. * A `MutantRunResult` no longer has to report back all tests that have run. Instead, it only reports the id of the test that killed the mutant (if the status is indeed `killed`). * A `MutantRunResult` no longer has to report back all tests that have run. Instead, it only reports the id of the test that killed the mutant (if the status is indeed `killed`). * Added a `toMutantRunResult` helper function that takes in a `dryRunResult` and can convert it to a `mutantRunResult`. The new API is published under `TestRunner2`, as well as a separate plugin type. Later we will remove the old test runner API and rename this new one to `TestRunner` proper.
Bumps [karma](https://github.com/karma-runner/karma) from 5.0.9 to 5.1.0. - [Release notes](https://github.com/karma-runner/karma/releases) - [Changelog](https://github.com/karma-runner/karma/blob/master/CHANGELOG.md) - [Commits](karma-runner/karma@v5.0.9...v5.1.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Bumps [eslint-plugin-prettier](https://github.com/prettier/eslint-plugin-prettier) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/prettier/eslint-plugin-prettier/releases) - [Changelog](https://github.com/prettier/eslint-plugin-prettier/blob/master/CHANGELOG.md) - [Commits](prettier/eslint-plugin-prettier@v3.1.3...v3.1.4) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
* fix(core): fix "too many open files" Fixes "too many open files" issue by limitting the number of concurrent file IO to 256 * One liner await * fix unit test * test(input-file): don't rely on os ordering * Remove `.only` * deterministic order of mutants
Bumps [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) from 6.11.0 to 6.12.0. - [Release notes](https://github.com/prettier/eslint-config-prettier/releases) - [Changelog](https://github.com/prettier/eslint-config-prettier/blob/master/CHANGELOG.md) - [Commits](prettier/eslint-config-prettier@v6.11.0...v6.12.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
add deprecation warning for function based config BREAKING CHANGE: exporting a function from stryker.conf.js is deprecated. Please export your config as an object instead, or use a stryker.conf.json file. Co-authored-by: Nico Jansen <jansennico@gmail.com>
Only add the JS header for files that have mutants. For example: don't add it in *.d.ts files.
Bumps [eslint](https://github.com/eslint/eslint) from 7.9.0 to 7.10.0. - [Release notes](https://github.com/eslint/eslint/releases) - [Changelog](https://github.com/eslint/eslint/blob/master/CHANGELOG.md) - [Commits](eslint/eslint@v7.9.0...v7.10.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Bumps [eslint-plugin-import](https://github.com/benmosher/eslint-plugin-import) from 2.22.0 to 2.22.1. - [Release notes](https://github.com/benmosher/eslint-plugin-import/releases) - [Changelog](https://github.com/benmosher/eslint-plugin-import/blob/master/CHANGELOG.md) - [Commits](import-js/eslint-plugin-import@v2.22.0...v2.22.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Correctly report the initial test run timing. For example: > `INFO DryRunExecutor Initial test run succeeded. Ran 502 tests in 2 seconds (net 2295 ms, overhead 572 ms)` Co-authored-by: root <root@SimonL03.corp.infosupport.com> Co-authored-by: Nico Jansen <jansennico@gmail.com>
Support shebang (i.e. `#!/usr/bin/env node`) in javascript files.
Add support for `--maxTestRunnerReuse` compiler flag (and equivalent in stryker.conf.json). With it, you can specify the maximum number that a test runner can be reused before the process is killed and a new one is created. The default is `0`, which means no maximum, just keep reusing. There are 2 use cases for this setting. 1. You have a memory leak in your tests that cannot be fixed (for example, a plugin in jest). You run with `--maxTestRunnerReuse 20`. 2. Your environment is dirty after running one time and you cannot do anything about it. You run with `--maxTestRunnerReuse 1`.
* Upgrade angular core to v9. * Update stryker.conf.json
Bumps [sinon](https://github.com/sinonjs/sinon) from 9.0.3 to 9.1.0. - [Release notes](https://github.com/sinonjs/sinon/releases) - [Changelog](https://github.com/sinonjs/sinon/blob/master/CHANGELOG.md) - [Commits](sinonjs/sinon@v9.0.3...v9.1.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Bumps [flatted](https://github.com/WebReflection/flatted) from 3.0.5 to 3.1.0. - [Release notes](https://github.com/WebReflection/flatted/releases) - [Commits](WebReflection/flatted@v3.0.5...v3.1.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Add a `MutantPlacer` for `SwitchCase` nodes. It places the mutants in the `consequence` part of the switch case. This can dramatically improve the performance of instrumenting large switch-case statements.
This:
```js
switch(foo){
case 'bar':
console.log('bar');
break;
case 'baz':
console.log('baz');
break;
}
```
Is now instrumented as:
```js
switch (foo) {
case stryMutAct_9fa48(1) ? \\"\\" : (stryCov_9fa48(1), 'bar'):
if (stryMutAct_9fa48(0)) {} else {
stryCov_9fa48(0);
console.log(stryMutAct_9fa48(2) ? \\"\\" : (stryCov_9fa48(2), 'bar'));
break;
}
case stryMutAct_9fa48(4) ? \\"\\" : (stryCov_9fa48(4), 'baz'):
if (stryMutAct_9fa48(3)) {} else {
stryCov_9fa48(3);
console.log(stryMutAct_9fa48(5) ? \\"\\" : (stryCov_9fa48(5), 'baz'));
break;
}
}
```
While previously the entire `SwitchCaseExpression` was duplicated for each empty switch case mutation.
Add support to configure `mutator.excludedMutations`. This was already supported in Stryker@<3, but not yet in Stryker4 beta. Instead of not generating the mutants, it generates them without placing them in the code. They get reported as `ignored` in your mutation testing report.  Also, add error message, ignore reason, and "killed by" test name in the "show more" modal dialog. 
Bumps [@jest/types](https://github.com/facebook/jest/tree/HEAD/packages/jest-types) from 26.3.0 to 26.5.2. - [Release notes](https://github.com/facebook/jest/releases) - [Changelog](https://github.com/facebook/jest/blob/master/CHANGELOG.md) - [Commits](https://github.com/facebook/jest/commits/v26.5.2/packages/jest-types) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
* Update the vuejs, react, angular and vue presets for the 4.0 release. * Remove `@stryker-mutator/core` from presets
Don't mutate `TSTypeParameterInstantiation` like `React.useState<true>()`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1514
Fixes #1946