Byte Code Dotnet Mutation Utility

Faultify provides a quick simple way to realize dotnet mutation testing at the byte code level. It imitates the bad programmer by deliberately introducing errors. A test is assumed to fail after an introduced mutation, the test is likely to be error-prone if it instead succeeds.

disclaimer: faultify is just released and bugs can be expected, please open a issue if you get any.

Getting Started

• Read the Technical Book
• Benchmark Statistics

Commandline Options

  -t, --testProjectName    Required. The path pointing to the test project project file.
  -r, --reportPath         The path were the report will be saved.
  -f, --reportFormat       (Default: json) Type of report to be generated, options: 'pdf', 'html', 'json'
  -p, --parallel           (Default: 1) Defines how many test sessions are ran at the same time.
  -l, --mutationLevel      (Default: Detailed) The mutation level indicating the test depth.
  
  --help                   Display this help screen.
  --version                Display version information.

Install / Run

dotnet tool install --global faultify --version 0.2.0
faultify -t YourTestProject.csproj -f html

This generates a 'HTML' report for the project 'YourTestProject.csproj' at the default executable location in the folder '/FaultifyOutput'.

Features

Functional Features

• Mutate Arithmetic (+, -, /, *, %) Operators.
• Mutate Assignment Expressions (+=, -=, /=, *=, %=, --, ++).
• Mutate Equivalence Operators (==, !=).
• Mutate Logical Operators (&&, ||).
• Change bitwise operators (^,|, &).
• Mutate Branching statements (if(condition), if(!condition))
• Mutate Variable Literals (true, false).
  • Support for other types.
• Mutate Constant Fields (string, number, boolean).
• Mutate Array initializations (Only arrays larger than 2 elements, all C# types)
• Build mutation report (HTML/PDF).
• Mutation test algorithm.

Non-Functional Features

• Cross-platform .net core compatibility.
• Nunit/Xunit/Msunit support via dotnet test.
• Runnable from console.
• All dotnet languages (F#, C#, visualbaisc) support.

Todo

• Implement member exclusion.
• Implement line number display
• Improve reporting functionality
• Implement assembly testing in memory

Application Preview

Solution Projects

Solution Item	Description
Faultify.Cli	Contains the faultify executable mutation dotnet utility.
Faultify.Analyze	Contains the code that analyzes and searches for mutations
Faultify.TestRunner	Contains the code that runs the mutation test process
Faultify.Report	Contains the code that generates a report
/Bechmarks	Contains a benchmark project and tests that can be used for mutation testing with faultify/stryker.