[Bug]: Storybook loads stories from node_modules with pnpm

[Methuselah96]

Describe the bug

When using pnpm, Stoybook seems to be loading stories from node_modules when the stories patterns could include node_modules of subdirectories. This seems to be the same issue as #18907 and #19446, but it does not seem to be fixed as of 7.6.3 and goes against the behavior described in #22873. In a large monorepo with lots of dependencies, this crashes the dev server and build with an out of memory error (but that's harder to reproduce):

<--- Last few GCs --->

[1987:0x67a8920]    61264 ms: Mark-Compact (reduce) 2033.9 (2082.7) -> 2033.1 (2082.7) MB, 2284.10 / 0.00 ms  (average mu = 0.104, current mu = 0.001) allocation failure; scavenge might not succeed
[1987:0x67a8920]    63550 ms: Mark-Compact (reduce) 2034.1 (2082.7) -> 2033.4 (2083.2) MB, 2284.16 / 0.00 ms  (average mu = 0.054, current mu = 0.001) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xc9e850 node::Abort() [node]
 2: 0xb720ff  [node]
 3: 0xec1a70 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0xec1d57 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 5: 0x10d3dc5  [node]
 6: 0x10ebc48 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0x10c1d61 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0x10c2ef5 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 9: 0x10a0466 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
10: 0x14fb386 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
11: 0x7f7377e99ef6 
Aborted (core dumped)
 ELIFECYCLE  Command failed with exit code 134.

To Reproduce

1. Clone https://github.com/Methuselah96/storybook-pnpm-issue
2. pnpm install
3. pnpm storybook
4. Observe errors that show it's trying to read stories from node_modules:

ERROR in ./node_modules/.pnpm/@storybook+react@7.6.3_react-dom@18.2.0_react@18.2.0_typescript@4.9.5/node_modules/@storybook/react/template/cli/ts-3-8/Button.stories.ts 1:12
Module parse failed: Unexpected token (1:12)
File was processed with these loaders:
 * ./node_modules/.pnpm/@storybook+builder-webpack5@7.6.3_esbuild@0.18.20_typescript@4.9.5_webpack-cli@5.1.4/node_modules/@storybook/builder-webpack5/dist/loaders/export-order-loader.js
 * ./node_modules/.pnpm/unplugin@1.5.1/node_modules/unplugin/dist/webpack/loaders/load.js
You may need an additional loader to handle the result of these loaders.

System

Storybook Environment Info:

  System:
    OS: Windows 11 10.0.22621
    CPU: (8) x64 Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  Binaries:
    Node: 20.9.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - C:\Program Files\nodejs\yarn.CMD
    npm: 10.1.0 - C:\Program Files\nodejs\npm.CMD <----- active
    pnpm: 8.11.0 - C:\Program Files\nodejs\pnpm.CMD
  Browsers:
    Edge: Chromium (119.0.2151.97)

Additional context

No response

[valentinpalkovic]

Until the issue is fixed, could you try the following workaround?:

Instead of
stories: ["../packages/**/*.mdx", "../packages/**/*.stories.@(js|jsx|mjs|ts|tsx)"],

Please try to use the following glob:
stories: ["../packages/(**|!(node_modules))/*.mdx", "../packages/(**|!(node_modules))/*.stories.@(js|jsx|mjs|ts|tsx)"],

I would appreciate any feedback

[Methuselah96]

Thanks for taking a look! Unfortunately, updating the glob pattern as suggested doesn't seem to fix it, however it does seem to change something.

I updated https://github.com/Methuselah96/storybook-pnpm-issue with that change. Now it's saying that it can't find any stories, despite there being stories in packages/a/src/stories/*.

I also used that updated pattern for my real-life project, and it still crashes with an OOM error.

Let me know if you need anything else!

[Methuselah96]

For more context, in my real-life project, if I narrow the glob patterns to be the src directory of each package (to explicitly exclude node_modules), then it does not crash with an OOM error. So narrowing the glob pattern does help, but the updated glob pattern from above doesn't seem to do the trick. I can explicitly list all the directories where stories can exist while carefully avoiding node_modules, but obviously I'd prefer to just include all directories with Storybook automatically excluding node_modules (which seems to be the intended behavior of #22873).

[shilman]

I suspect that this is a windows-specific bug since #22873 was only tested on mac

[valentinpalkovic]

@shilman I don't think so. As far as I can see #22873 was only fixing the issue for the Vite builder and not for the Webpack builder. I have reproduced the issue on Mac already.

[shilman]

Ah webpack 🤦‍♂️