Programmatic way to run Vitest

[TeunLamers]

Clear and concise description of the problem

Is there a way to run Vitest programmatically. For StrykerJS we are investigating support StrykerJS. Ideally we would like to be able to set an global environment right before each test run and be able to recover a global environment variable.

Vitest support is currently the most upvoted issue for us. stryker-mutator/stryker-js#3465

Suggested solution

// Code below is pseudo-code to illustrate our needed API. 
import { Vitest } from 'vitest';

const vitest = new Vitest();

// runInBand in order to share global variables with the test environment
await vitest.readConfig({ overrides: 'here', runInBand: true, concurrency: 1 });

// register global hook: beforeEach, needed to determine coverage per test. We can also say a test file is a test instead (and use the filename as test name).
vitest.beforeEach((test) => {
  global.__stryker__.currentTest = test.name; 
});

// Load the test files without running the tests. Or run the test once, but be able to rerun them without reloading the files
await vitest.loadFiles();
const results = await vitest.run();
// Results include the test names or test filenames including the test results (passed or failed).

// Grab coverage from a global variable 
const { mutationCoverage } = global.__stryker__;
// Determine test coverage per mutation.

// Test mutant 1:
global.__stryker__.activeMutation = 1;
const resultMutantOne = await vitest.run({ testFilter: ['..array of test names to run for mutant 1'] });

// Test mutant 2:
global.__stryker__.activeMutation = 2;
const resultMutantOne = await vitest .run({ testFilter: ['..array of test names to run for mutant 2'] });

Alternative

No response

Additional context

We are okay with using a non public API for now and we are willing to do the work 😅
#524

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[sheremet-va]

There is non-documented (because it's not stable) way to programatically run Vitest:

import { startVitest } from 'vitest/node'
await startVitest('test', ['filename'])

I think this is how WallabyJS uses it. Currently it returns only true/false, if run was successful. But it will return Vitest (also available from vitest/node) in #2072

[sheremet-va]

You can also use createVitest that will return you Vitest context:

import { createVitest } from 'vitest/node'
const ctx = await createVitest('test')
await ctx.start(['filename'])

All tests results are located in ctx.state.getFiles(), but you can also use custom reporter to track tests completion.

[TeunLamers]

Thank you for the fast reply. We are working on a prototype! We'll get back to you for any questions.

[sheremet-va]

There is a small page describing the public API in the documentation: https://vitest.dev/advanced/api.html