[Bug]: Explicit action mocks are not always reset between tests in test runner (race condition)

[bodograumann]

Describe the bug

In storybook 7 we could use argTypes: { onClick: true } on meta to register the event as an action and have it automatically mocked in play functions.
Now in storybook 8 we instead use args: { onClick: fn() }.

In our project we now had the problem, that the mocks are not cleared between tests and so negative assertions on events were failing.
It took me a while to create a minimal reproduction, because in a naive setup this does not occur right away.
Turns out it is some kind of race condition. Sometimes it works, sometimes it does not. I had to preload fn in preset.js to cause the race condition in the reproduction + run the tests on a built storybook instead of a live server.

To Reproduce

1. git clone https://github.com/bodograumann/storybook-8-fn-clear
2. cd storybook-8-fn-clear
3. yarn
4. npx storybook build --test
5. npx http-server storybook-static -a 127.0.0.1 --port 6005
6. In a new terminal: npx test-storybook --url http://localhost:6005
7. Retry 6. a few times. It fails most of the time (~89/100) and sometimes succeeds.

System

npx storybook@latest info

Storybook Environment Info:
(node:44360) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

  System:
    OS: Linux 6.7 Arch Linux
    CPU: (20) x64 12th Gen Intel(R) Core(TM) i7-12700H
    Shell: 5.9 - /usr/bin/zsh
  Binaries:
    Node: 21.7.0 - /usr/bin/node
    Yarn: 1.22.21 - /usr/bin/yarn
    npm: 10.5.0 - /usr/bin/npm <----- active
    pnpm: 8.15.2 - /usr/bin/pnpm
  npmPackages:
    @storybook/addon-designs: ~7.0.9 => 7.0.9
    @storybook/addon-essentials: ^8.0.0 => 8.0.0
    @storybook/addon-interactions: ^8.0.0 => 8.0.0
    @storybook/blocks: ^8.0.0 => 8.0.0
    @storybook/mdx2-csf: ^1.1.0 => 1.1.0
    @storybook/test: ^8.0.0 => 8.0.0
    @storybook/test-runner: ^0.17.0 => 0.17.0
    @storybook/vue3: ^8.0.0 => 8.0.0
    @storybook/vue3-vite: ^8.0.0 => 8.0.0
    chromatic: ^11.0.0 => 11.0.8
    eslint-plugin-storybook: ^0.8.0 => 0.8.0
    storybook: ^8.0.0 => 8.0.0
  npmGlobalPackages:
    @storybook/cli: 7.6.8

Additional context

Original discussion, where clearing the mocks was discussed: #23649

Not sure how hard this is to solve, but I'll try and work around the issue by manually clearing the mocks, where tests are currently failing. Of course actually failing tests might be obscured by this bug, because I cannot manually clear the mocks in every play function.

[kasperpeulen]

Really weird, this seems to solve it:

import { fn, restoreAllMocks } from "@storybook/test";

const _dummy = fn();
_dummy.mockClear();

export default {
  loaders: [
    () => {
      restoreAllMocks();
    },
  ],
};

But that is exactly what we do by default. Investigating.

[kasperpeulen]

Ah we have already solved this in 8.1.0-alpha.1. Let's release 8.0.1 soon @vanessayuenn

[vanessayuenn]

Thanks @kasperpeulen! I will close this issue for now then. @bodograumann please feel free to reopen if this issue persists in 8.1-alpha. This fix should also be patched to 8.0.1 soon (probably next week) -- stay tuned!

[bodograumann]

v8.0.1 has been released, but I cannot find a fix related to this issue in the changelog, @vanessayuenn.
Was it included and it's just not obvious, or was in not backported yet?

[bodograumann]

I tried installing 8.0.1 and afaict the issue is gone. So thanks everyone! <3

[kasperpeulen]

@bodograumann The issue was that we were installing multiple version of storybook/test, which is fixed in:
#26411