test.each name serialization has changed to be less useful, impacting snapshots.

[Jason3S]

Describe the bug

When migrating from 0.31.4 to 0.34.1, it appears that the serializer used to format test names has changed, impacting snapshots and test result readability.

Example snapshot change:

- exports[`Validate cli > app success 'suggest' run with [ 'suggest', 'café', …(2) ] 1`] = `[]`;
+ exports[`Validate cli > app success 'suggest' run with [ Array(4) ] 1`] = `[]`;

Array(4) is not very useful. This leads to a lot of duplicate test names. It also makes figuring out which test failed painful. I understand that due to test order, this should not be an issue with matching up snapshots, but it has lead to a LOT of churn with respect to .snap files.

I hope this gets fixed, because identifying a failed test out of hundreds can be painful if they all result in the same displayed name and snapshot identifier.

Reproduction

import { describe, expect, test } from 'vitest';

describe('test.each name serialization', () => {
    test.each`
        name        | params
        ${'array'}  | ${['one', 'two', 'three', 'four', 'five']}
        ${'array'}  | ${['red', 'green', 'blue', 'yellow', 'orange']}
        ${'array'}  | ${['a', 'b', 'c', 'd', 'e']}
        ${'object'} | ${{ one: 'one', two: 'two', three: 'three', four: 'four' }}
        ${'object'} | ${{ one: 1, two: 2, three: 3, four: 4 }}
        ${'object'} | ${{ one: 'first', two: 'second', three: 'third', four: 'fourth' }}
        ${'object'} | ${{ a: 'a', b: 'b', c: 'c', d: 'd' }}
    `('Example issue of name serialization $name $params', ({ name, params }) => {
        expect({ name, params }).toMatchSnapshot();
    });
});

Diff of snapshot going from 0.31.4 to 0.34.1:

diff --git a/__snapshots__/t.test.ts.snap b/__snapshots__/t.test.ts.snap
index d53a3506a..f23637455 100644
--- a/__snapshots__/t.test.ts.snap
+++ b/__snapshots__/t.test.ts.snap
@@ -13,7 +13,7 @@ exports[`test.each name serialization > Example issue of name serialization 'arr
 }
 `;
 
-exports[`test.each name serialization > Example issue of name serialization 'array' [ 'one', 'two', 'three', 'four', …(1) ] 1`] = `
+exports[`test.each name serialization > Example issue of name serialization 'array' [ Array(5) ] 1`] = `
 {
   "name": "array",
   "params": [
@@ -26,7 +26,7 @@ exports[`test.each name serialization > Example issue of name serialization 'arr
 }
 `;
 
-exports[`test.each name serialization > Example issue of name serialization 'array' [ 'red', 'green', 'blue', …(2) ] 1`] = `
+exports[`test.each name serialization > Example issue of name serialization 'array' [ Array(5) ] 2`] = `
 {
   "name": "array",
   "params": [
@@ -39,19 +39,19 @@ exports[`test.each name serialization > Example issue of name serialization 'arr
 }
 `;
 
-exports[`test.each name serialization > Example issue of name serialization 'object' { a: 'a', b: 'b', c: 'c', d: 'd' } 1`] = `
+exports[`test.each name serialization > Example issue of name serialization 'object' { Object (one, two, ...) } 1`] = `
 {
   "name": "object",
   "params": {
-    "a": "a",
-    "b": "b",
-    "c": "c",
-    "d": "d",
+    "four": "four",
+    "one": "one",
+    "three": "three",
+    "two": "two",
   },
 }
 `;
 
-exports[`test.each name serialization > Example issue of name serialization 'object' { one: 'first', two: 'second', …(2) } 1`] = `
+exports[`test.each name serialization > Example issue of name serialization 'object' { Object (one, two, ...) } 2`] = `
 {
   "name": "object",
   "params": {
@@ -63,14 +63,14 @@ exports[`test.each name serialization > Example issue of name serialization 'obj
 }
 `;
 
-exports[`test.each name serialization > Example issue of name serialization 'object' { one: 'one', two: 'two', …(2) } 1`] = `
+exports[`test.each name serialization > Example issue of name serialization 'object' { a: 'a', b: 'b', c: 'c', d: 'd' } 1`] = `
 {
   "name": "object",
   "params": {
-    "four": "four",
-    "one": "one",
-    "three": "three",
-    "two": "two",
+    "a": "a",
+    "b": "b",
+    "c": "c",
+    "d": "d",
   },
 }
 `;

System Info

System:
    OS: Windows 10 10.0.22621
    CPU: (24) x64 12th Gen Intel(R) Core(TM) i9-12900K
    Memory: 9.54 GB / 31.73 GB
  Binaries:
    Node: 18.15.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - C:\Program Files\nodejs\yarn.CMD
    npm: 9.5.0 - C:\Program Files\nodejs\npm.CMD
    pnpm: 8.6.9 - ~\AppData\Local\pnpm\pnpm.CMD
  Browsers:
    Edge: Spartan (44.22621.2134.0), Chromium (115.0.1901.203)
    Internet Explorer: 11.0.22621.1
  npmPackages:
    @vitest/coverage-istanbul: ^0.34.1 => 0.34.1
    vite: ^4.4.9 => 4.4.9
    vitest: ^0.34.1 => 0.34.1

Used Package Manager

pnpm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[ghiscoding]

prior to v032.1 they were using concordance to show diff but it had perf issues with large set of tests output and they went back to using Jest diff output (using pretty-format) in PR #3582 (see release v0.32.1) which is probably when it changed for you. So it's likely that your diff changed at that time, to be sure I would try the v0.32.x

Also since they're back to using Jest diff output, I know that Jest v29 changed their default snapshot format (see their Jest 29 blogpost) and it looks like it shows the old format on your side. I think there is a flag you change in your settings to show the new snapshot format, I would have to search for it...

yes here is the link:
https://vitest.dev/guide/snapshot.html#_2-printbasicprototype-is-default-to-false

hope that is helpful :)

[sheremet-va]

@ghiscoding actually, this PR talks about test name formating. It is different now because instead of using node's built-in util module we are using loupe that comes with chai now for better browser compatibility.

[hi-ogawa]

I was going back some commits and maybe this commit 4f6c134 changed something? (which might not be related node-util vs loupe change)

It looks like currently loupe's truncate option is not used for objDisplay unless users explicitly provide chaiConfig.truncateThreshold even though documentation suggests the default is 40 https://vitest.dev/config/#chaiconfig-truncatethreshold.

In such case, the "untruncated" loupe output is checked against default 40 length and manually truncated to [ Array(n) ] etc... if it exceeds. Also currently raw string is not truncated.

I made a repro to show the effect of chaiConfig.truncateThreshold:

• Without chaiConfig.truncateThreshold:

https://stackblitz.com/edit/vitest-dev-vitest-5cqlhe?file=test%2Frepro.test.ts

 ✓ test/repro.test.ts (9)
   ✓ '012345678901234567890123456789' (length = 30)
   ✓ '0123456789012345678901234567890123456789' (length = 40)
   ✓ '01234567890123456789012345678901234567890123456789' (length = 50)
   ✓ [ 'one', 'two', 'three' ]
   ✓ [ 'one', 'two', 'three', 'four' ]
   ✓ [ Array(5) ]
   ✓ { one: 1, two: 2, three: 3 }
   ✓ { one: 1, two: 2, three: 3, four: 4 }
   ✓ { Object (one, two, ...) }

• With explicit chaiConfig.truncateThreshold: 40:

https://stackblitz.com/edit/vitest-dev-vitest-mpfgjn?file=test%2Frepro.test.ts

 ✓ test/repro.test.ts (7)
   ✓ '012345678901234567890123456789' (length = 30)
   ✓ '0123456789012345678901234567890123456…' (length = 40)
   ✓ '0123456789012345678901234567890123456…' (length = 50)
   ✓ [ 'one', 'two', 'three' ]
   ✓ [ 'one', 'two', 'three', 'four' ]
   ✓ [ 'one', 'two', 'three', 'four', …(1) ]
   ✓ [ 'one', 'two', 'three', 'four', …(2) ]
   ✓ { one: 1, two: 2, three: 3 }
   ✓ { one: 1, two: 2, three: 3, four: 4 }
   ✓ { one: 1, two: 2, three: 3, …(2) }

[Jason3S]

@hi-ogawa,

Thank you! I would never have found that setting.