Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storybook won't start when used in a pnpm workspace #237

Closed
robcaldecott opened this issue Feb 11, 2022 · 9 comments
Closed

Storybook won't start when used in a pnpm workspace #237

robcaldecott opened this issue Feb 11, 2022 · 9 comments

Comments

@robcaldecott
Copy link

Note that I reported this on the main Storybook repo but was told to report it here:

storybookjs/storybook#17342 (comment)

Describe the bug
I am experimenting with pnpm as its monorepo support is excellent. However, when using Storybook with storybook-builder-vite in a workspace it will not start and instead displays the following error:

info => Loading presets
ERR! Error: [vite-plugin-mdx] "@mdx-js/react" must be installed
ERR!     at resolveImport (/Users/caldecor/Projects/pnpm-storybook/node_modules/.pnpm/artifactory.keyloop.com+vite-plugin-mdx@3.5.10_@mdx-js+mdx@1.6.22+vite@2.7.13/node_modules/vite-plugin-mdx/dist/imports.js:29:23)
<snip>

On the face of it this looks like a dependency resolution issue as the same repo will work OK using npm8 or yarn1 workspaces.

To Reproduce
You will need pnpm installed (npm install -g pnpm).

Clone the following repo: https://github.com/robcaldecott/pnpm-storybook and then:

pnpm install
cd packages/components
pnpm storybook

System

  System:
    OS: macOS 10.15.7
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  Binaries:
    Node: 16.13.2 - ~/.nvm/versions/node/v16.13.2/bin/node
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm
  Browsers:
    Chrome: 97.0.4692.99
    Firefox: 95.0.2
    Safari: 14.1.2
  npmPackages:
    @storybook/addon-actions: ^6.4.14 => 6.4.14 
    @storybook/addon-essentials: ^6.4.14 => 6.4.14 
    @storybook/addon-links: ^6.4.14 => 6.4.14 
    @storybook/react: ^6.4.14 => 6.4.14 

Additional context
There is a proposed workaround by creating a .npmrc file in the root of the repo that looks like this:

node-linker=hoisted

However this causes pnpm some issues and seems like a last resort.

@robcaldecott
Copy link
Author

FWIW when using hoisting it exposes a bizarre pnpm issue!

pnpm/pnpm#4284

@IanVS
Copy link
Member

IanVS commented Feb 11, 2022

Have you taken a look at #55? Is this a different issue from that one?

@robcaldecott
Copy link
Author

Could be. All the errors I'm saying are from vite-plugin-mdx

@IanVS
Copy link
Member

IanVS commented Feb 17, 2022

There are some workarounds in that other issue, have you tried them out?

@ufon
Copy link

ufon commented Feb 22, 2022

  async viteFinal(config, { configType }) {
    const { dirname } = require('path');
    
    // https://github.com/eirslett/storybook-builder-vite/issues/55
    config.root = dirname(require.resolve('storybook-builder-vite'));
    // config.server.fsServe = {};
    return config;
  },

Working for me.

@zeyuri
Copy link

zeyuri commented Mar 13, 2022

@ufon is this is still working for you? Which version of the plugin are you using?

Can you please share a working example of this? I'm struggling very hard to make this work.

@IanVS
Copy link
Member

IanVS commented Mar 17, 2022

Closing as a duplicate of #55. Definitely something on our radar, and we'd love if anyone was able to find a way to solve this without requiring shame.

@IanVS IanVS closed this as completed Mar 17, 2022
dhess added a commit to hackworthltd/primer-app that referenced this issue May 25, 2022
This is a wide-ranging change in which we drop `yarn2nix` (and Yarn
itself) in favor using `pnpm` directly, with no Nix layer in-between
the `pnpm` lockfile and the packages we use for builds. This means
we're giving up on most of what Nix provides -- here we only use it to
ensure that everyone is using the same version of `pnpm`, sourced from
`nipxkgs` via our `nix develop` shell.

I'm doing this reluctantly, but `yarn2nix` is falling further and
further behind the Nodejs ecosystem, it always feels like we're just
one major package version bump away from everything breaking, and
there's no viable replacement on the horizon for `yarn2nix` that I can
see. Nix is not really buying us much in this repo to begin with, so
we don't lose that much with this change. Hopefully this leads to
better DX going forward overall.

Note that the `README.md` is heavily edited to replace all the `yarn`
commands with the equivalent `pnpm` commands (where they exist; we
have lost some functionality by dropping Yarn, see below), and I
wouldn't be surprised if there are some discrepancies in the
documentation vs the reality. Please file an issue if you notice any.

A few more items of note:

* I have bumped our `nixpkgs` pin here, but only to get a more recent
version of `pnpm`. I have not bumped any Nodejs package versions.

* `pnpm` is more strict about dependencies, so I've had to add a few
that Yarn was magically finding for us.

* We've lost the nice `wsrun watch` and `wsrun watch:storybook`
commands that would run the entire project in `watch` (HMR) mode with
a single command. This is because `wsrun` is designed to work with
Yarn workspaces only, and doesn't work with `pnpm` workspaces. As far
as I can tell, there's no equivalent to `wsrun` for `pnpm`, which
means that if you want the run the frontend or our Storybook in HMR
mode and automatically pick up changes made to the other packages in
the repo upon which the frontend/Storybook depend, you'll need to open
multiple shells and run a `pnpm watch` command in each one,
simultaneously.

* We do not implement the push-to-Chromatic CI step here, as I want to
test just building the app before we get into trying to build
Storybook in CI. (However, I have been able to successfully build and
use Storybook in my local shell with these changes.)

* Speaking of Storybook, we have to do some ugly hacks to make it work
with `pnpm`. Many people have trouble with this and there are a litany
of issues, many with different workarounds that are sometimes
successful, sometimes not; see:

pnpm/pnpm#4268
storybookjs/storybook#13428
https://github.com/storybookjs/builder-vite/blob/782261437b3cb3e2f3d013cd37f1faa14863f205/packages/builder-vite/README.md#installation
storybookjs/builder-vite#55
storybookjs/builder-vite#237
https://github.com/IanVS/vite-storybook-pnpm-shame
https://github.com/jdk2pq/vite-storybook-pnpm-vue

Until Storybook fix their various issues, this looks like the proper
"fix", but it will require quite a bit of work, I suspect:

pnpm/pnpm#4268 (comment)

However, in the interest of time, I've chosen this option, which is
the least bad workaround I've seen that worked for me:

storybookjs/storybook#13428 (comment)
dhess added a commit to hackworthltd/primer-app that referenced this issue May 25, 2022
This is a wide-ranging change in which we drop `yarn2nix` (and Yarn
itself) in favor using `pnpm` directly, with no Nix layer in-between
the `pnpm` lockfile and the packages we use for builds. This means
we're giving up on most of what Nix provides -- here we only use it to
ensure that everyone is using the same version of `pnpm`, sourced from
`nipxkgs` via our `nix develop` shell.

I'm doing this reluctantly, but `yarn2nix` is falling further and
further behind the Nodejs ecosystem, it always feels like we're just
one major package version bump away from everything breaking, and
there's no viable replacement on the horizon for `yarn2nix` that I can
see. Nix is not really buying us much in this repo to begin with, so
we don't lose that much with this change. Hopefully this leads to
better DX going forward overall.

Note that the `README.md` is heavily edited to replace all the `yarn`
commands with the equivalent `pnpm` commands (where they exist; we
have lost some functionality by dropping Yarn, see below), and I
wouldn't be surprised if there are some discrepancies in the
documentation vs the reality. Please file an issue if you notice any.

A few more items of note:

* I have bumped our `nixpkgs` pin here, but only to get a more recent
version of `pnpm`. I have not bumped any Nodejs package versions.

* `pnpm` is more strict about dependencies, so I've had to add a few
that Yarn was magically finding for us.

* We've lost the nice `wsrun watch` and `wsrun watch:storybook`
commands that would run the entire project in `watch` (HMR) mode with
a single command. This is because `wsrun` is designed to work with
Yarn workspaces only, and doesn't work with `pnpm` workspaces. As far
as I can tell, there's no equivalent to `wsrun` for `pnpm`, which
means that if you want the run the frontend or our Storybook in HMR
mode and automatically pick up changes made to the other packages in
the repo upon which the frontend/Storybook depend, you'll need to open
multiple shells and run a `pnpm watch` command in each one,
simultaneously.

* We do not implement the push-to-Chromatic CI step here, as I want to
test just building the app before we get into trying to build
Storybook in CI. (However, I have been able to successfully build and
use Storybook in my local shell with these changes.)

* Speaking of Storybook, we have to do some ugly hacks to make it work
with `pnpm`. Many people have trouble with this and there are a litany
of issues, many with different workarounds that are sometimes
successful, sometimes not; see:

pnpm/pnpm#4268
storybookjs/storybook#13428
https://github.com/storybookjs/builder-vite/blob/782261437b3cb3e2f3d013cd37f1faa14863f205/packages/builder-vite/README.md#installation
storybookjs/builder-vite#55
storybookjs/builder-vite#237
https://github.com/IanVS/vite-storybook-pnpm-shame
https://github.com/jdk2pq/vite-storybook-pnpm-vue

Until Storybook fix their various issues, this looks like the proper
"fix", but it will require quite a bit of work, I suspect:

pnpm/pnpm#4268 (comment)

However, in the interest of time, I've chosen this option, which is
the least bad workaround I've seen that worked for me:

storybookjs/storybook#13428 (comment)
dhess added a commit to hackworthltd/primer-app that referenced this issue May 25, 2022
This is a wide-ranging change in which we drop `yarn2nix` (and Yarn
itself) in favor using `pnpm` directly, with no Nix layer in-between
the `pnpm` lockfile and the packages we use for builds. This means
we're giving up on most of what Nix provides -- here we only use it to
ensure that everyone is using the same version of `pnpm`, sourced from
`nipxkgs` via our `nix develop` shell.

I'm doing this reluctantly, but `yarn2nix` is falling further and
further behind the Nodejs ecosystem, it always feels like we're just
one major package version bump away from everything breaking, and
there's no viable replacement on the horizon for `yarn2nix` that I can
see. Nix is not really buying us much in this repo to begin with, so
we don't lose that much with this change. Hopefully this leads to
better DX going forward overall.

Note that the `README.md` is heavily edited to replace all the `yarn`
commands with the equivalent `pnpm` commands (where they exist; we
have lost some functionality by dropping Yarn, see below), and I
wouldn't be surprised if there are some discrepancies in the
documentation vs the reality. Please file an issue if you notice any.

A few more items of note:

* I have bumped our `nixpkgs` pin here, but only to get a more recent
version of `pnpm`. I have not bumped any Nodejs package versions.

* `pnpm` is more strict about dependencies, so I've had to add a few
that Yarn was magically finding for us.

* We've lost the nice `wsrun watch` and `wsrun watch:storybook`
commands that would run the entire project in `watch` (HMR) mode with
a single command. This is because `wsrun` is designed to work with
Yarn workspaces only, and doesn't work with `pnpm` workspaces. As far
as I can tell, there's no equivalent to `wsrun` for `pnpm`, which
means that if you want the run the frontend or our Storybook in HMR
mode and automatically pick up changes made to the other packages in
the repo upon which the frontend/Storybook depend, you'll need to open
multiple shells and run a `pnpm watch` command in each one,
simultaneously.

* We do not implement the push-to-Chromatic CI step here, as I want to
test just building the app before we get into trying to build
Storybook in CI. (However, I have been able to successfully build and
use Storybook in my local shell with these changes.)

* Speaking of Storybook, we have to do some ugly hacks to make it work
with `pnpm`. Many people have trouble with this and there are a litany
of issues, many with different workarounds that are sometimes
successful, sometimes not; see:

pnpm/pnpm#4268
storybookjs/storybook#13428
https://github.com/storybookjs/builder-vite/blob/782261437b3cb3e2f3d013cd37f1faa14863f205/packages/builder-vite/README.md#installation
storybookjs/builder-vite#55
storybookjs/builder-vite#237
https://github.com/IanVS/vite-storybook-pnpm-shame
https://github.com/jdk2pq/vite-storybook-pnpm-vue

Until Storybook fix their various issues, this looks like the proper
"fix", but it will require quite a bit of work, I suspect:

pnpm/pnpm#4268 (comment)

However, in the interest of time, I've chosen this option, which is
the least bad workaround I've seen that worked for me:

storybookjs/storybook#13428 (comment)
dhess added a commit to hackworthltd/primer-app that referenced this issue May 25, 2022
This is a wide-ranging change in which we drop `yarn2nix` (and Yarn
itself) in favor using `pnpm` directly, with no Nix layer in-between
the `pnpm` lockfile and the packages we use for builds. This means
we're giving up on most of what Nix provides -- here we only use it to
ensure that everyone is using the same version of `pnpm`, sourced from
`nipxkgs` via our `nix develop` shell.

I'm doing this reluctantly, but `yarn2nix` is falling further and
further behind the Nodejs ecosystem, it always feels like we're just
one major package version bump away from everything breaking, and
there's no viable replacement on the horizon for `yarn2nix` that I can
see. Nix is not really buying us much in this repo to begin with, so
we don't lose that much with this change. Hopefully this leads to
better DX going forward overall.

Note that the `README.md` is heavily edited to replace all the `yarn`
commands with the equivalent `pnpm` commands (where they exist; we
have lost some functionality by dropping Yarn, see below), and I
wouldn't be surprised if there are some discrepancies in the
documentation vs the reality. Please file an issue if you notice any.

A few more items of note:

* I have bumped our `nixpkgs` pin here, but only to get a more recent
version of `pnpm`. I have not bumped any Nodejs package versions.

* `pnpm` is more strict about dependencies, so I've had to add a few
that Yarn was magically finding for us.

* We've lost the nice `wsrun watch` and `wsrun watch:storybook`
commands that would run the entire project in `watch` (HMR) mode with
a single command. This is because `wsrun` is designed to work with
Yarn workspaces only, and doesn't work with `pnpm` workspaces. As far
as I can tell, there's no equivalent to `wsrun` for `pnpm`, which
means that if you want the run the frontend or our Storybook in HMR
mode and automatically pick up changes made to the other packages in
the repo upon which the frontend/Storybook depend, you'll need to open
multiple shells and run a `pnpm watch` command in each one,
simultaneously.

* We do not implement the push-to-Chromatic CI step here, as I want to
test just building the app before we get into trying to build
Storybook in CI. (However, I have been able to successfully build and
use Storybook in my local shell with these changes.)

* Speaking of Storybook, we have to do some ugly hacks to make it work
with `pnpm`. Many people have trouble with this and there are a litany
of issues, many with different workarounds that are sometimes
successful, sometimes not; see:

pnpm/pnpm#4268
storybookjs/storybook#13428
https://github.com/storybookjs/builder-vite/blob/782261437b3cb3e2f3d013cd37f1faa14863f205/packages/builder-vite/README.md#installation
storybookjs/builder-vite#55
storybookjs/builder-vite#237
https://github.com/IanVS/vite-storybook-pnpm-shame
https://github.com/jdk2pq/vite-storybook-pnpm-vue

Until Storybook fix their various issues, this looks like the proper
"fix", but it will require quite a bit of work, I suspect:

pnpm/pnpm#4268 (comment)

However, in the interest of time, I've chosen this option, which is
the least bad workaround I've seen that worked for me:

storybookjs/storybook#13428 (comment)
dhess added a commit to hackworthltd/primer-app that referenced this issue May 26, 2022
This is a wide-ranging change in which we drop `yarn2nix` (and Yarn
itself) in favor using `pnpm` directly, with no Nix layer in-between
the `pnpm` lockfile and the packages we use for builds. This means
we're giving up on most of what Nix provides -- here we only use it to
ensure that everyone is using the same version of `pnpm`, sourced from
`nipxkgs` via our `nix develop` shell.

We also now run most CI on GitHub Actions, rather than Buildkite. See
the notes below for a detailed justification.

I'm doing this reluctantly, but `yarn2nix` is falling further and
further behind the Nodejs ecosystem, it always feels like we're just
one major package version bump away from everything breaking, and
there's no viable replacement on the horizon for `yarn2nix` that I can
see. Nix is not really buying us much in this repo to begin with, so
we don't lose that much with this change. Hopefully this leads to
better DX going forward overall.

Note that the `README.md` is heavily edited to replace all the `yarn`
commands with the equivalent `pnpm` commands (where they exist; we
have lost some functionality by dropping Yarn, see below), and I
wouldn't be surprised if there are some discrepancies in the
documentation vs the reality. Please file an issue if you notice any.

A few more items of note:

* I have bumped our `nixpkgs` pin here, but only to get a more recent
version of `pnpm`. I have not bumped any Nodejs package versions.

* `pnpm` is more strict about dependencies, so I've had to add a few
that Yarn was magically finding for us.

* We've lost the nice `wsrun watch` and `wsrun watch:storybook`
commands that would run the entire project in `watch` (HMR) mode with
a single command. This is because `wsrun` is designed to work with
Yarn workspaces only, and doesn't work with `pnpm` workspaces. As far
as I can tell, there's no equivalent to `wsrun` for `pnpm`, which
means that if you want the run the frontend or our Storybook in HMR
mode and automatically pick up changes made to the other packages in
the repo upon which the frontend/Storybook depend, you'll need to open
multiple shells and run a `pnpm watch` command in each one,
simultaneously.

* With this change, we build the frontend and test it (or at least,
will test it, once we have some tests to run!) on GitHub Actions, not
on Buildkite. The reasons for this are that while the Nix Buildkite
plugin is wonderful, and works much better than any Nix-related GitHub
Action that I've seen, there are no comparable Buildkite plugins for
installing a particular version of `pnpm`, caching `pnpm` builds
across multiple CI runs, or deploying to Chromatic. Therefore, if we
stuck with Buildkite CI for this repo, we would need to implement all
that functionality ourselves, probably via bespoke scripts. That's a
waste of time when there are existing vendor-provided GitHub Actions
which do all of those things for us. We still do have a few minor
checks & some Nix shell caching CI jobs on Buildkite, but most likely
those will eventually go away and be replaced by equivalent GitHub
Actions, so that we only need to look in one place for CI results.

* On that note, we do not implement the push-to-Chromatic CI step
here, as I want to test just building the app before we get into
trying to build Storybook in CI. (However, I have been able to
successfully build and use Storybook in my local shell with these
changes.) Support for this is coming in a subsequent commit.

* Speaking of Storybook, we have to do some ugly hacks to make it work
with `pnpm`. Many people have trouble with this and there are a litany
of issues, many with different workarounds that are sometimes
successful, sometimes not; see:

pnpm/pnpm#4268
storybookjs/storybook#13428
https://github.com/storybookjs/builder-vite/blob/782261437b3cb3e2f3d013cd37f1faa14863f205/packages/builder-vite/README.md#installation
storybookjs/builder-vite#55
storybookjs/builder-vite#237
https://github.com/IanVS/vite-storybook-pnpm-shame
https://github.com/jdk2pq/vite-storybook-pnpm-vue

Until Storybook fix their various issues, this looks like the proper
"fix", but it will require quite a bit of work, I suspect:

pnpm/pnpm#4268 (comment)

However, in the interest of time, I've chosen this option, which is
the least bad workaround I've seen that worked for me:

storybookjs/storybook#13428 (comment)
dhess added a commit to hackworthltd/primer-app that referenced this issue May 26, 2022
This is a wide-ranging change in which we drop `yarn2nix` (and Yarn
itself) in favor using `pnpm` directly, with no Nix layer in-between
the `pnpm` lockfile and the packages we use for builds. This means
we're giving up on most of what Nix provides -- here we only use it to
ensure that everyone is using the same version of `pnpm`, sourced from
`nipxkgs` via our `nix develop` shell.

We also now run most CI on GitHub Actions, rather than Buildkite. See
the notes below for a detailed justification.

I'm doing this reluctantly, but `yarn2nix` is falling further and
further behind the Nodejs ecosystem, it always feels like we're just
one major package version bump away from everything breaking, and
there's no viable replacement on the horizon for `yarn2nix` that I can
see. Nix is not really buying us much in this repo to begin with, so
we don't lose that much with this change. Hopefully this leads to
better DX going forward overall.

Note that the `README.md` is heavily edited to replace all the `yarn`
commands with the equivalent `pnpm` commands (where they exist; we
have lost some functionality by dropping Yarn, see below), and I
wouldn't be surprised if there are some discrepancies in the
documentation vs the reality. Please file an issue if you notice any.

A few more items of note:

* I have bumped our `nixpkgs` pin here, but only to get a more recent
version of `pnpm`. I have not bumped any Nodejs package versions.

* `pnpm` is more strict about dependencies, so I've had to add a few
that Yarn was magically finding for us.

* We've lost the nice `wsrun watch` and `wsrun watch:storybook`
commands that would run the entire project in `watch` (HMR) mode with
a single command. This is because `wsrun` is designed to work with
Yarn workspaces only, and doesn't work with `pnpm` workspaces. As far
as I can tell, there's no equivalent to `wsrun` for `pnpm`, which
means that if you want the run the frontend or our Storybook in HMR
mode and automatically pick up changes made to the other packages in
the repo upon which the frontend/Storybook depend, you'll need to open
multiple shells and run a `pnpm watch` command in each one,
simultaneously.

* With this change, we build the frontend and test it (or at least,
will test it, once we have some tests to run!) on GitHub Actions, not
on Buildkite. The reasons for this are that while the Nix Buildkite
plugin is wonderful, and works much better than any Nix-related GitHub
Action that I've seen, there are no comparable Buildkite plugins for
installing a particular version of `pnpm`, caching `pnpm` builds
across multiple CI runs, or deploying to Chromatic. Therefore, if we
stuck with Buildkite CI for this repo, we would need to implement all
that functionality ourselves, probably via bespoke scripts. That's a
waste of time when there are existing vendor-provided GitHub Actions
which do all of those things for us. We still do have a few minor
checks & some Nix shell caching CI jobs on Buildkite, but most likely
those will eventually go away and be replaced by equivalent GitHub
Actions, so that we only need to look in one place for CI results.

* On that note, we do not implement the push-to-Chromatic CI step
here, as I want to test just building the app before we get into
trying to build Storybook in CI. (However, I have been able to
successfully build and use Storybook in my local shell with these
changes.) Support for this is coming in a subsequent commit.

* Speaking of Storybook, we have to do some ugly hacks to make it work
with `pnpm`. Many people have trouble with this and there are a litany
of issues, many with different workarounds that are sometimes
successful, sometimes not; see:

pnpm/pnpm#4268
storybookjs/storybook#13428
https://github.com/storybookjs/builder-vite/blob/782261437b3cb3e2f3d013cd37f1faa14863f205/packages/builder-vite/README.md#installation
storybookjs/builder-vite#55
storybookjs/builder-vite#237
https://github.com/IanVS/vite-storybook-pnpm-shame
https://github.com/jdk2pq/vite-storybook-pnpm-vue

Until Storybook fix their various issues, this looks like the proper
"fix", but it will require quite a bit of work, I suspect:

pnpm/pnpm#4268 (comment)

However, in the interest of time, I've chosen this option, which is
the least bad workaround I've seen that worked for me:

storybookjs/storybook#13428 (comment)
@wilbsy
Copy link

wilbsy commented Nov 29, 2022

.npmrc file in root directory with the following lines solved for me:

auto-install-peers = true
node-linker=hoisted

@palerdot
Copy link

  async viteFinal(config, { configType }) {
    const { dirname } = require('path');
    
    // https://github.com/eirslett/storybook-builder-vite/issues/55
    config.root = dirname(require.resolve('storybook-builder-vite'));
    // config.server.fsServe = {};
    return config;
  },

Working for me.

@ufon solution works for me. I just had to tweak the require to @storybook/builder-vite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants