Skip to content
This repository has been archived by the owner on May 14, 2020. It is now read-only.

Set up mutation tests #11

Merged
merged 4 commits into from
Apr 1, 2020
Merged

Conversation

thewilkybarkid
Copy link
Contributor

Configures and runs Stryker, a mutation tester.

Configures and runs [Stryker](https://stryker-mutator.io/), a mutation tester.
@thewilkybarkid thewilkybarkid added the ⚙️ CI Changes to the continuous integration configuration files and scripts label Mar 31, 2020
@thewilkybarkid thewilkybarkid requested review from a team as code owners March 31, 2020 10:08
@@ -66,6 +66,10 @@ test: export TARGET = dev
test: ## Run the tests
${DOCKER_COMPOSE} run --rm app npm run test

test\:mutation: export TARGET = dev
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these additional explicit exports required. Doesn't the makefile set these up at the top?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It forces TARGET to be dev for test:mutation (ie TARGET=prod make test:mutation continues to work as TARGET=prod is ignored).

"mutate": [
"src/**/*.ts"
],
"mutator": "typescript",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the time or mutation/tests number budget here? In theory it could run forever but there must be a limit in place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is how to make it generate more mutations, but the code may be too simple for that. Adding a couple of operations in src/app.ts raises the number of mutants:

Ran 1.60 tests per mutant on average.
----------|---------|----------|-----------|------------|----------|---------|
File      | % score | # killed | # timeout | # survived | # no cov | # error |
----------|---------|----------|-----------|------------|----------|---------|
All files |   25.00 |        1 |         0 |          3 |        0 |       1 |
 app.ts   |   25.00 |        1 |         0 |          3 |        0 |       1 |
----------|---------|----------|-----------|------------|----------|---------|
14:03:00 (17) ERROR MutationTestReportCalculator Final mutation score 25.00 under breaking threshold 100, setting exit code to 
1 (failure).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libero/article-store#167 shows more usage.

Co-Authored-By: Giorgio Sironi <accounts@giorgiosironi.com>
],
"testRunner": "jest",
"thresholds": {
"break": 100
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explored whether to set high thinking that the 80% default wouldn't be matching. Doesn't change anything.

Makefile Outdated Show resolved Hide resolved
@thewilkybarkid thewilkybarkid merged commit 3d9f9e2 into libero:master Apr 1, 2020
@thewilkybarkid thewilkybarkid deleted the mutation-tests branch April 1, 2020 08:22
Copy link

@erkannt erkannt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
⚙️ CI Changes to the continuous integration configuration files and scripts
Projects
None yet
3 participants