ci: run tests sequentially

[sheremet-va]

Description

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
• Ideally, include a test that fails without this PR but passes with it.
• Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

• Run the tests with pnpm test:ci.

Documentation

• If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

• Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

[netlify]

✅ Deploy Preview for fastidious-cascaron-4ded94 canceled.

Name	Link
🔨 Latest commit	74a49ca
🔍 Latest deploy log	https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/663ba4e07d972c0008e479aa

[AriPerkkio]

test/core is failing randomly without any error logs. Is #5678 on this branch?

@vitest/test-core : test test/core
  test/core test$ vitest
  (node:14941) [DEP0180] DeprecationWarning: fs.Stats constructor is deprecated.
  (Use `node --trace-deprecation ...` to show where the warning was created)
   RUN  v1.6.0 /home/runner/work/vitest/vitest/test/core
        API started at http://localhost:3023/
   ✓ |forks| test/injector-esm.test.ts  (42 tests) 62ms
   ✓ |vmThreads| test/injector-esm.test.ts  (42 tests) 72ms
   ✓ |threads| test/injector-esm.test.ts  (42 tests) 74ms
   ✓ |vmThreads| test/expect.test.ts  (21 tests) 49ms
  test/core test: Failed
  /home/runner/work/vitest/vitest/test/core:
   ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  @vitest/test-core@ test: `vitest`
  Exit status 129

[sheremet-va]

Is #5678 on this branch?

Yes, these failings are why I opened that PR. No idea why it's failing yet.

[sheremet-va]

is failing randomly without any error logs

Do you think it might be because we have 3 different pools that have their own min/max worker count?

[AriPerkkio]

Oh right, those pools are run in parallel. The failing linux CIs have 2 CPUs so each pool is using just one worker. But still there's main thread + 3 pool threads/processes running.

[AriPerkkio]

Actually I can reproduce this issue locally with Node v22.1.0 - really weird. Removing vmThreads pool fixes the issue.

Screen.Recording.2024-05-06.at.20.20.17.mov

[AriPerkkio]

Isolated the crashing test to pnpm run test --run --project=vmThreads injector-mock.test.ts. Changing that file to contain just the content below is enough to crash:

import { test } from 'vitest'
import { hoistMocks } from '../../../packages/vitest/src/node/hoistMocks'

test.only('Helloo', () => {
  console.log(hoistMocks)
})

[sheremet-va]

Isolated the crashing test to pnpm run test --run --project=vmThreads injector-mock.test.ts. Changing that file to contain just the content below is enough to crash:

import { test } from 'vitest'
import { hoistMocks } from '../../../packages/vitest/src/node/hoistMocks'

test.only('Helloo', () => {
  console.log(hoistMocks)
})

Interesting. The only thing I can think of is that it imports parseAst which relies on a native swc module 🤔

[sheremet-va]

The only thing I can think of is that it imports parseAst

Oh, it doesn't actually import it, the plugin provides it 🤔

[sheremet-va]

Looks like importing generateCodeFrame can cause this. Digging deeper 🔍

[sheremet-va]

Almost always fails during import of vite:

creating module : /vitest/node_modules/.pnpm/vite@5.2.6_@types+node@20.11.5/node_modules/vite/dist/node/index.js

🔍

vitest/packages/vitest/src/runtime/external-executor.ts

Line 232 in 0227e8c

case 'module':

[sheremet-va]

Looks like it always fails here:

vitest/packages/vitest/src/runtime/vm/file-map.ts

Line 11 in 0227e8c

const source = readFileSync(path, 'utf-8')

No error, just exit with 129 code - interestingly, it kills even the main thread

[AriPerkkio]

Could you try if v22.0.0 works on CI? Locally it doesn't crash like v22.1.0 does. This could help to pinpoint the root cause. 👀

vitest/.github/workflows/ci.yml

Line 42 in bcccce6

node_version: [18, 20, 22]

-        node_version: [18, 20, 22]
+        node_version: [18, 20, 22.0.0]

[sheremet-va]

Could you try if v22.0.0 works on CI? Locally it doesn't crash like v22.1.0 does. This could help to pinpoint the root cause. 👀

vitest/.github/workflows/ci.yml

Line 42 in bcccce6

node_version: [18, 20, 22]

-        node_version: [18, 20, 22]
+        node_version: [18, 20, 22.0.0]

Actually, I am not even sure why we test against Node 22, it's not LTS yet. We usually wait for it to be LTS or close to it before we add it to CI because it's prone to bugs like this,

[AriPerkkio]

Sure, we could drop it until its LTS plans have been announced and add it back some weeks/months before that.