Percy Visual Regression fails with timeouts when initializeWorker is called

[IsidorusHispalensis]

We are using MSW to run visual regression using Percy and recent update of npm packages leads to timeouts in E2E tests.

Package name	Version
@storybook/react	6.3.10
msw	0.35.0
msw-storybook-addon	1.4.0

Unfortunately the repo is private and I am not able to share it but I can create an empty one if you want, steps to reproduce:

1. Create an empty folder and initialise it with npm/yarn
2. Add latest react
3. Call 'npx sb init' to install Storybook with default stories
4. Install @percy/cli @percy/storybook (you will need a free Percy account to run tests)
5. Run percy tests -- it will pass
6. Install MSW Storybook Addon and run percy tests again -- it will fail with timeouts

In interactive mode I can see Chromium is doing multiple page reloads when MSW is attached, and removing initializeWorker() from preview.js clearly solves the problem. Any help to get directions where to continue digging is extremely appreciated

[oscard0m]

@wwilsman @Robdel12 Tagging you since I see you are the most active in @percy/storybook. Any clue con what could be the issue? Is Percy compatible with Service Workers (MSW uses one for mocking API responses)?

Sorry if I tagged the wrong Percy team members here, I just pretend to give some visibility to the issue. I'm not sure if it's more related to msw addon or to Percy

[jefffohl]

Also having this issue. My guess is that this is a Percy issue. Guessing that Percy doesn't support service workers?

[Robdel12]

👋🏼 It's likely causing issues with asset discovery. Percy's SDK needs to request the assets to save them. Can you share debug logs? https://docs.percy.io/docs/debugging-sdks#debug-vs-verbose-logging

My guess is asset discovery can't request/save assets because they're being mocked or blocked.

[jefffohl]

Thanks @Robdel12 - I just started working with Percy today. I will look for logs and share them.

[jefffohl]

@Robdel12 here is the output of running percy with storybook in debug mode. There are a number of types of errors. I am not sure which are the relevant ones.

The process for obtaining this output is:

1. Launch Storybook to run on port 6006
2. Storybook is configured to launch msw through this in Storybook's launcher, preview.js:

const { worker } = require('../src/mocks/browser');
worker.start();

msw provides mocked data and images which are then used in the stories.
3. Run Percy: npx percy storybook http://localhost:6006 --debug

percy-log.txt.zip

EDIT: btw, this is a React component library, which uses the MS Fluent UI component library.

[FezVrasta]

My tests hang with the official puppeteer storyshots addon, Storybook itself works fine when I run it, but it times out when I run the tests.

[oscard0m]

I think I'm experiencing the same error. When starting Storybook in local with MSW add-on integrated all seems to be working perfectly and the expected requests are handled by the service worker.

When putting Percy in the middle is when MSW does not play well with Storybook apparently.

Steps to reproduce it:

1. build-storybook -s public -> ✅
2. percy storybook ./storybook-static --verbose --debug -> 💣 Percy is not able to generate snapshots

Dependencies

	version
Node	14.18.0
@percy-cli	1.0.0-beta.68
@percy/storybook	4.0.3
msw-storybook-addon	1.4.1
msw	0.35
@storybook/core	6.3.8

preview.js

Click to expand!

import "../src/styles/fonts.css";
import { initialize, mswDecorator } from 'msw-storybook-addon';

// Initialize MSW
initialize({
serviceWorker: {
  url: `${process.env.PUBLIC_URL || ""}/mockServiceWorker.js`,
},
onUnhandledRequest: 'bypass'
});

// Provide the MSW addon decorator globally
export const decorators = [mswDecorator];

export const parameters = {
actions: { argTypesRegex: "^on[A-Z].*" },
viewport: {
  defaultViewport: "mobile2",
},
controls: {
  matchers: {
    color: /(background|color)$/i,
    date: /Date$/,
  },
},
};

MyComponent.stories.js

Click to expand!

import React from "react";
import Upload from "../../routes/upload";
import { handlers } from "../../mocks/handlers";

const Meta = {
title: "Routes/Upload",
component: Upload,
};

export default Meta;

const Template = () => {
return (
  <div style={{ height: "95vh", display: "flex" }}>
    <Upload />
  </div>
);
};

export const Primary = Template.bind({});
Primary.parameters = {
msw: handlers,
layout: "fullscreen",
};

_{*Tried with Primary.parameters.percy.enableJavaScript: true and getting the same issue}

---

Logs

Attaching logs from my use-case with --debug and --verbose flags enabled:

I guess is the same error as @jefffohl has. Repeated output for each of the resources:

Encountered an error processing resource: http://localhost:55971/static/media/cloud.ebe94e3c.svg (7ms)
[percy:core:discovery] Error: Protocol error (Network.getResponseBody): No resource with given identifier found
    at /Users/XXX/dev/simplabs/experteer/cv2profile/node_modules/@percy/core/dist/session.js:85:16
    at new Promise (<anonymous>)
    at Session.send (/Users/XXX/dev/simplabs/experteer/cv2profile/node_modules/@percy/core/dist/session.js:83:12)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Object.request.response.buffer (/Users/XXX/dev/simplabs/experteer/cv2profile/node_modules/@percy/core/dist/network.js:218:22)
    at async Network.onRequestFinished (/Users/XXX/dev/simplabs/experteer/cv2profile/node_modules/@percy/core/dist/discovery.js:78:32)
    at async Session.<anonymous> (/Users/XXX/dev/simplabs/experteer/cv2profile/node_modules/@percy/core/dist/network.js:241:7)

Complete output:

percy-logs.txt

[jefffohl]

One possible workaround could be using the http-middleware module. Run the same mocks using a node server instead of a service worker when running Percy tests: https://github.com/mswjs/http-middleware

[Robdel12]

Likely what will need to happen. Since this is a network interception library, it looks like it's doing its job by intercepting network requests. Percy also watches requests being made in the browser to capture them, so it sounds like this library is preventing that from happening (https://docs.percy.io/docs/debugging-sdks#how-sdks-work)

[oscard0m]

One possible workaround could be using the http-middleware module. Run the same mocks using a node server instead of a service worker when running Percy tests: https://github.com/mswjs/http-middleware

I'm going to take a look into this. Thanks for sharing @jefffohl.

Likely what will need to happen. Since this is a network interception library, it looks like it's doing its job by intercepting network requests. Percy also watches requests being made in the browser to capture them, so it sounds like this library is preventing that from happening (https://docs.percy.io/docs/debugging-sdks#how-sdks-work)

Thanks @Robdel12 for the information and detailed answer on how Percy works with assets of a web app. I learnt a lot reading that explanation. Still, there is something not clear to me yet:

Even the Service Worker provided by MSW captures all the requests and handles the ones configured by the provided handlers, the rest of the requests are bypassed as far as I understand. For example, checking Network's activity when loading Storybook, I see this:

For my personal knowledge and learning opportunity:

• Are these requests really going to through network?
• Should Percy be detecting this kind of requests?

[filipw01]

Does anyone have a sample repo with a workaround solution with http-middleware?
Are there other workarounds for this?
Are there any plans to fix it either from the MSW addon or Percy cli perspective?

[Robdel12]

One workaround could be to add a custom user agent config to Percy's asset discovery browser: https://docs.percy.io/docs/cli-configuration#discovery then when that UA is detected in storybook you would disable this addon.

Should Percy be detecting this kind of requests?

Percy is doing request interception at the browser level via CDP. So any network request that runs through the browser will be seen & processed by the SDK.

[andrzej-woof]

Does anyone have a sample repo with a workaround solution with http-middleware?

I've came up with the following code placed in .storybook/middleware.js

const cors = require('cors');
const bodyParser = require('body-parser');
const { createMiddleware } = require('@mswjs/http-middleware');
const handlersQueries = require('./mocks/handler');

const mockServiceWorkerMiddleware = router => {
  router.use(cors());
  router.use(bodyParser.urlencoded({
    extended: true
  }));
  router.use(bodyParser.json());  
  router.use(createMiddleware(
    ...handlersQueries
  ));
};

module.exports = mockServiceWorkerMiddleware;

Few notes:

1. This will of course only work with storybook server, not with the static build
2. It's good to include cors() middleware if you want to set up dedicated mock server instead of using the storybook one. Here it's probably not necessary at all as mock shares host:port with storybook
3. It was vital to include body-parser middleware, took me quite some time to figure out why @http/msw-middleware wasn't handling GQL requests at all, and (obviously) req.body was undefined

This solution does seem to work with percy-storybook, unlike the service worker mock

[oscard0m]

Should Percy be detecting this kind of requests?

Percy is doing request interception at the browser level via CDP. So any network request that runs through the browser will be seen & processed by the SDK.

I'm missing knowledge on this but seems that Puppeteer and Playwright are struggling with this as well:

• Requests are not intercepted when cached by Service Worker puppeteer/puppeteer#4041
• Support ServiceWorkers puppeteer/puppeteer#2634
• [Feature] support route aka request interception for service workers microsoft/playwright#1090

@Robdel12 is Percy based on Puppeteer or Playwright? Or it is interacting directly with CDP?

---

In comments, they mention some Chromium issues which could mean CDP is not receiving this king of requests right now?

• [Feature] support route aka request interception for service workers microsoft/playwright#1090 (comment)
• [Feature] support route aka request interception for service workers microsoft/playwright#1090 (comment)
• [Feature] support route aka request interception for service workers microsoft/playwright#1090 (comment)

[oscard0m]

Does anyone have a sample repo with a workaround solution with http-middleware?

I've came up with the following code placed in .storybook/middleware.js

const cors = require('cors');
const bodyParser = require('body-parser');
const { createMiddleware } = require('@mswjs/http-middleware');
const handlersQueries = require('./mocks/handler');

const mockServiceWorkerMiddleware = router => {
  router.use(cors());
  router.use(bodyParser.urlencoded({
    extended: true
  }));
  router.use(bodyParser.json());  
  router.use(createMiddleware(
    ...handlersQueries
  ));
};

module.exports = mockServiceWorkerMiddleware;

Few notes:

1. This will of course only work with storybook server, not with the static build
2. It's good to include cors() middleware if you want to set up dedicated mock server instead of using the storybook one. Here it's probably not necessary at all as mock shares host:port with storybook
3. It was vital to include body-parser middleware, took me quite some time to figure out why @http/msw-middleware wasn't handling GQL requests at all, and (obviously) req.body was undefined

This solution does seem to work with percy-storybook, unlike the service worker mock

Is there a way to use @mswjs/http-middleware but to set the handlers at Story level? I would like to apply a certain handler for a concrete set of Stories and not share all the list of mocks for all the Stories.

@andrzej-kodify

[andrzej-woof]

Is there a way to use @mswjs/http-middleware but to set the handlers at Story level? I would like to apply a certain handler for a concrete set of Stories and not share all the list of mocks for all the Stories.

@oscard0m nothing obvious that I can think of here :(
As a workaround I'd probably try and play around with parameters to base the mock response on something that comes with request, or maybe referer header 🤔

[oscard0m]

Also, @andrzej-kodify, does your example cover mock requests of external URL's or only localhost/? I'm having problems with that right now but I'm guessing .storybook/middleware only intercepts localhost requests. Is this correct?

[andrzej-woof]

@oscard0m only localhost, you could try to modify request url before it's sent or add an entry hooking given domain to 127.0.0.1 in the system hosts file to capture other traffic

[oscard0m]

Just for the ones interested:

Until this issue moves forward, the temporal solution we are using in our company is storybook-addon-mock.

It does not rely on a Service Worker so Percy seems to like it.

[joekur]

I am interested in a resolution for this issue. We are moving to MSW more widely at my organization, but we have a lot of apps that use percy storybook.

This doesn't seem to be an issue when using percy with other techs (e.g. capybara or webdriverio) - is there an explanation for that?

It also looks like playwright made progress on this, maybe that can help inform a direction here: microsoft/playwright#1090

[ninadbstack]

@joekur I went through this thread right now. We would have to check the viability of supporting MSW based stories. As you have already reported this in a support ticket, we would look into this and update on it.

[nilshah98]

Hello everyone,
I wanted to provide a brief update on the latest developments. Percy has recently launched new SDKs, offering support for utilizing MSW to mock API responses in conjunction with Storybook. You can find the documentation for this feature here.

Feel free to give it a try and share your feedback at https://github.com/percy/percy-storybook.

[ninadbstack]

@oscard0m can we close this discussion as with official support for the plugin its resolved ?

[oscard0m]

@oscard0m can we close this discussion as with official support for the plugin its resolved ?

We are using Chromatic for our projects so I can't check, but I assume it can be closed if there's official support.

[ninadbstack]

@IsidorusHispalensis / @oscard0m can you close the issue as a reporter/maintainer ?

[oscard0m]

@IsidorusHispalensis / @oscard0m can you close the issue as a reporter/maintainer ?

I don't have permissions to do it.