jest-snapshot: Display change counts in annotation lines

[pedrottimark]

Summary

The number of change lines helps to interpret differences when a snapshot assertion fails:

• only delete lines (that is, in snapshot but not in received)
• only insert lines
• equal number of delete and insert lines
• almost equal number of delete and insert lines
• quite unequal number of delete and insert lines

Similar to benefit of summary report following a test run, except these precede the comparison.

In addition to the includeChangeCounts option from #8881 here are supporting changes:

1. Call functions directly from jest-diff instead of indirectly via jest-matcher-utils as prerequisite for color options in a future pull request, and also swap where two snapshot transformations are called:

   • Move removeExtraLineBreaks from _toMatchSnapshot function to match method, because it is a private detail of snapshot formatting
   • Move and rename unescape function from match method to printDiffOrStringified function, because its use depends on how the report displays the snapshot (see 2)

2. Adjust changes in Unescape serialized snapshots for diffing #2852 to unescape double quote marks and also backslashes but only when the report displays string differences, and if both of the following:

   • snapshot looks like default serialization of a string (otherwise an array or object can have strings which ought to be enclosed in double quote marks and have escapes)
   • received is string which has default serialization

3. With change counts as a clue to interpret edge cases when the snapshot or received is empty string and its counterpart is multiline string, display as comparison lines instead of labeled lines; which reverses a previous decision:

   • negative: takes a second thought to interpret 0 lines as empty string
   • positive: emphasizes analogy from snapshot review in console to Changed Files on GitHub
   • positive: solves alignment problem for multi-line strings discussed in jest-snapshot: Highlight substring differences when matcher fails, part 3 #8569 (review)
   • positive: display strings unescaped, also if both strings are one line (see 2)

Next steps

1. What do y’all think about a follow-up pull request to display change counts in differences when toBe, toEqual, toStrictEqual, toHaveProperty, toMatchObject assertions fail?

2. Change confusing colors in snapshot differences, but not exactly as requested in Diff colors should show green for insert, red for delete #5430

3. Ignore indentation in serialization of received objects and by heuristic in snapshots to remove distracting differences, see Optionally hide whitespace changes in diff #8626

Test plan

package	test file	changes
e2e	failures	updated 2 snapshots
e2e	toMatchSnapshot	replaced 1 old toMatch assertion with 2 new
e2e	watchModeUpdateSnapshot	updated 1 snapshot
jest-matcher-utils	printDiffOrStringify	replaced 1 old test (which had obsolete negative assertion) with 3 new
jest-snapshot	printDiffOrStringified	updated 23 17 and added 4 snapshots; added 3 6 tests

See also pictures in 4 following comments: baseline is at left and improved is at right

[pedrottimark]

In the following 5 pictures, the only difference is the change counts

The changes are insertions in the received value:

The changes are replacements:

This comparison will improve in future pull request to ignore indentation:

Some changed lines have substring differences:

Unlike other assertions, snapshots do not “know” the expected type:

[scotthovestadt]

Code looks solid to me. Not 100% sold on the way it's displayed and want opinions.

[scotthovestadt]

For this case, where it's a string that was 1 line and will again be 1 line, this information is pretty useless. Can we not display it?

[pedrottimark]

Yes, I will deal with comparison of two one-line strings as a special case for the concise labeled format, because it seems likely for toThrowErrorMatchingSnapshot assertions.

[jeysal]

Maybe with a clearer labeling (other comment thread) it's actually fine and worth keeping for consistency?

[pedrottimark]

Tim, good point about prioritizing consistency higher than a more concise special case.

If most reports have differences when snapshot assertions fail, and if next pull request changes the colors to be less confusing, some people might learn to scan quickly which console output is from snapshots versus other assertions, and more intuitively choose which decision path.

For example, the e2e tests for this pull request had 3 snapshots to update and 1 toMatch assertion which I needed to rewrite as 4 assertions.

To test this theory: think fast, what do you need to do for each of the following:

Expected: "bar"
Received: "foo"

Snapshot: "bar"
Received: "foo"

[jeysal]

I think I know what you mean, if there's lots of (correct) test failures I usually try to change non-snapshot tests first and then update snapshots once they're the only things still failing. Not sure how that's related to this comment thread but I'd definitely appreciate making it easier to tell them apart.

[pedrottimark]

See pictures in #8982 (comment) of concise label format if neither string is multiline

[pedrottimark]

For default serialization in concise format, improved does not unescape string differences:

See if you can interpret the meaning

For default string serialization, keep the concise label format:

The improved picture at the right is possible with custom raw string serialization:

Because the received is multiline string, display as comparison lines:

For default string serialization, keep the concise label format:

The improved picture at the right is possible with custom raw string serialization:

[pedrottimark]

In the following 3 pictures of edge cases, improved does not unescape quoted strings:

baseline code never unescaped backslashes and improved does not in object serialization:

baseline (above) always unescaped double quotes but improved (below) doesn’t in object serialization:

do not display differences for a few received types like number

the expected snapshot string '1 \'\n2 "\n3 \\' has escaped characters:

[pedrottimark]

The last picture illustrates the edge case of change from quoted string in default serialization to unquoted string in raw serialization, like recent chores #8957 #8959 #8976

[codecov-io]

Codecov Report

Merging #8982 into master will increase coverage by 0.15%.
The diff coverage is 95.74%.

@@            Coverage Diff             @@
##           master    #8982      +/-   ##
==========================================
+ Coverage    63.9%   64.05%   +0.15%     
==========================================
  Files         277      277              
  Lines       11652    11676      +24     
  Branches     2860     2863       +3     
==========================================
+ Hits         7446     7479      +33     
+ Misses       3576     3572       -4     
+ Partials      630      625       -5

Impacted Files	Coverage Δ
packages/jest-snapshot/src/State.ts	0% <0%> (ø)	⬆️
packages/jest-snapshot/src/print.ts	100% <100%> (+29.62%)	⬆️
packages/jest-snapshot/src/utils.ts	95.18% <100%> (+0.18%)	⬆️
packages/jest-diff/src/printDiffs.ts	100% <100%> (ø)	⬆️
packages/jest-snapshot/src/index.ts	27.39% <50%> (-1.47%)	⬇️
packages/jest-matcher-utils/src/index.ts	83.22% <0%> (+0.64%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 630350a...96389e7. Read the comment docs.

[pedrottimark]

If neither quoted string is multiline, then restore a previous improvement to display with labels:

Small change suggests either update after clean up message or fragile test (for example, v8 versus Gecko) so replace toThrowErrorMatchingSnapshot with toThrow(regexp) assertion:

Either regression or change if received has no common substring with expected:

Reasons why not to display as unescaped strings without double quote marks:

• empty string looks confusing
• content of string might look like serialization of another type, like number 17

Possible regression if received message is empty string:

Rare edge case for expected snapshot to be empty string:

However, for custom raw string serialization, always display as diff comparison lines.

[jeysal]

@scotthovestadt wanna take a look again to see if you like it in the current state?

[pedrottimark]

And what do y’all think about a follow-up pull request to display change counts for other assertions?

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.