test.each's behaviour deviates from jest's

[jfparadis-appomni]

Describe the bug

As noted previously in #1461, there are significant differences in test.each() between vitest and jest.

Reproduction

1. When invoking test.each() with an array containing null, e.g. test.each([null]), the value will be processed as an array containing null ([null]) on Jest and converted to a null on Vitest.

Vitest used to return an empty array in that scenario, but this was changed in #1462) and is enforced in this test:

test.each([
  null,
  [null],
])('null is null', (value) => {
  expect(value).toBe(null)
})

https://github.com/vitest-dev/vitest/blob/main/test/core/test/each.test.ts#L11

However, the same test will fail in Jest:

 FAIL  core/test/each.test.ts
  ●  null is null
    expect(received).toBe(expected) // Object.is equality

    Expected: null
    Received: [null]

      43 |       [null],
      44 |     ])('null is null', (value) => {
    > 45 |       expect(value).toBe(null)
         |                     ^
      46 |     })
      47 |
      48 |     it.each([true, false])(

      at core/test/each.test.ts:45:21

Test Suites: 1 failed, 0 passed, 1 total
Tests:       1 failed, 0 skipped, 0 passed, 1 total

2. In addition, empty arrays are treated as empty arrays in Jest and undefined in Vitest:

test.each([
  true, false,
  0, 1234,
  new Date(),
  [], {}, () => {},
])('primitives', (value) => {
  console.log(value, typeof value);
});

On Jest, the output is:

      true boolean
      false boolean
      0 number
      1234 number
      2022-08-15T21:52:20.000Z object
      [] object // <-- Difference
      {} object
      [Function (anonymous)] function

On Vitrest, the output is:

      true boolean
      false boolean
      0 number
      1234 number
      2022-08-15T21:52:20.000Z object
      undefined undefined // <-- Difference
      {} object
      [Function (anonymous)] function

System Info

Binaries:
    npm: 8.11.0 - ~/.nvm/versions/node/v16.16.0/bin/npm
  Browsers:
    Chrome: 104.0.5112.79
    Safari: 15.6
  npmPackages:
    vite: 3.0.x => 3.0.7 
    vitest: 0.21.x => 0.21.1

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[bartoszgolebiowski]

Should we align with jest behavior?

[williamthorsen]

Jest's behaviour is exactly what I'd expect, even if I didn't know anything about Jest.

What's the rationale for converting [] to undefined and [null] to null?

[bartoszgolebiowski]

I am waiting for confirmation from core team. I think we should align to jest behavior. @sheremet-va

[sheremet-va]

I am waiting for confirmation from core team. I think we should align to jest behavior. @sheremet-va

let's align the behavior.

[williamthorsen]

Clarification of Jest's behaviour (v29.0.3). This wasn't entirely as expected.

These tests pass:

test.each([
  null,
])('value is null', (value) => {
  expect(value).toBeNull()
});

test.each([
  [null, null],
  [null, null],
])('if all cases are arrays of equal length, treats array elements as arguments', (value) => {
  // typeof value: null
  expect(value).toBeNull()
})

These tests fail:

test.each([
  null,
  [null], // <-- FAIL: value is `[null]`
])('null value or null item is null', (value) => {
  // typeof value: null | any[]
  expect(value).toBeNull()
})

test.each([
  [null], // <-- FAIL: value is `[null]`
  null,
])('null value or null item is null', (value) => {
  // typeof value: null | any[]
  expect(value).toBeNull()
})

// ** FAIL: Test times out **
test.each([
  [null],
  [null, null],
])('if all cases are arrays, treats array elements as arguments', (value, _value?) => {
  expect(value).toBeNull()
})