Visual testing with Cypress

[masakudamatsu]

Read https://docs.cypress.io/guides/tooling/visual-testing.html#Functional-vs-visual-testing for how to do it.

[masakudamatsu]

We tried Cypress Image Snapshot

• Official: https://www.npmjs.com/package/cypress-image-snapshot
• Docs: https://medium.com/norwich-node-user-group/visual-regression-testing-with-cypress-io-and-cypress-image-snapshot-99c520ccc595

But I keep getting this error: "The value of "offset" is out of range. It must be >= 0 and <= 13113564. Received -6792"

The error was reported in cypress-io/cypress#2034 (with the help of Gleb Bahmutov)

[masakudamatsu]

It seems visual regression testing requires some paid software such as Applitools

• See https://blog.kevinlamping.com/visual-regression-testing-is-stupid/

[masakudamatsu]

See my tweet https://twitter.com/masa_kudamatsu/status/1273263197047279616?s=21 on how to deal with the errors

[masakudamatsu]

We've figured out how to take the full-page snapshots

• See Screenshot fullpage jaredpalmer/cypress-image-snapshot#78 (comment)

We've created shortcut CLI commands to run cypress-image-snapshot

• npm run test:visual to run visual-tests.js only with Cypress CLI
• npm run test:update to update the snapshots

We only take the full-page snapshots

• This goes against the best practice of visual regression testing
  • https://blog.kevinlamping.com/visual-regression-testing-is-stupid/
  • https://docs.cypress.io/guides/tooling/visual-testing.html#DOM-state
• But we're interested in the overall layout, not each single component.
• We'll see if there's any inconvenience.

We take snapshots at each of the breakpoints as well as the narrowest screen (320px)

Snapshots are committed with Git LFS.

[masakudamatsu]

We also disable the image-snapshots if we run Cypress with Chrome.

• See https://glebbahmutov.com/blog/open-source-visual-testing-of-components/#local-workflow

[masakudamatsu]

For a summary of how to set up visual-testing with cypress-image-snapshot, see https://www.evernote.com/l/AFMjgtLhI5ZO757i757ke27x5Eal9I2xABE