Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try pnpm #37324

Closed
wants to merge 9 commits into from
Closed

Try pnpm #37324

wants to merge 9 commits into from

Conversation

youknowriad
Copy link
Contributor

@youknowriad youknowriad commented Dec 13, 2021

Related to #36142 and #37318. Alternative to #36041.

@youknowriad
Copy link
Contributor Author

youknowriad commented Dec 13, 2021

There's a weird typescript issue here that is causing most CI jobs to fail.

Error: packages/components/src/flyout/styles.ts(15,14): error TS2742: The inferred type of 'FlyoutContentView' cannot be named without a reference to '.pnpm/reakit-utils@0.15.2_react-dom@17.0.1+react@17.0.1/node_modules/reakit-utils/ts/types'. This is likely not portable. A type annotation is necessary.

@sirreal @gziolo any idea why this happens and if there's a workaround.

@gziolo
Copy link
Member

gziolo commented Dec 14, 2021

I'm looking into addressing some of the issues I encountered when trying to switch to npm 8 that might also pop up in this branch. See #37364 (React types) and #37365 (Redux). I have a longer list of tweaks to explore and I hope it will unblock our efforts so we can pick between npm 8 and pnpm.

@sirreal
Copy link
Member

sirreal commented Dec 14, 2021

I'm not really sure what the issue is, I'm not familiar with pnpm or Reakit. It seems to be related to this issue microsoft/TypeScript#42873

The message says "A type annotation is necessary," providing one seems to fix the problem:

diff --git a/packages/components/src/flyout/styles.ts b/packages/components/src/flyout/styles.ts
index b18cf6524d..3ac211d808 100644
--- a/packages/components/src/flyout/styles.ts
+++ b/packages/components/src/flyout/styles.ts
@@ -1,7 +1,7 @@
 /**
  * External dependencies
  */
-import styled from '@emotion/styled';
+import styled, { StyledComponent } from '@emotion/styled';
 // eslint-disable-next-line no-restricted-imports
 import { Popover as ReakitPopover } from 'reakit';
 
@@ -12,7 +12,9 @@ import { Card, CardBody } from '../card';
 import * as ZIndex from '../utils/z-index';
 import CONFIG from '../utils/config-values';
 
-export const FlyoutContentView = styled( ReakitPopover )`
+export const FlyoutContentView: StyledComponent<
+	React.ComponentPropsWithoutRef< typeof ReakitPopover >
+> = styled( ReakitPopover )`
 	z-index: ${ ZIndex.Flyout };
 	box-sizing: border-box;
 	opacity: 0;

(I think that's correct, but I'm not familiar with emotion or Reakit 🙂 )

cc: @ciampo who may be able to check my work

@ciampo
Copy link
Contributor

ciampo commented Dec 14, 2021

cc: @ciampo who may be able to check my work

Thank you for the ping Jon! I am not entirely sure if we need to exclude ref (cc @sarayourfriend ) and/or if reakit already exports the type for the Popover pros, which could potentially avoid calling ComponentPropsWithoutRef in the first place (cc @diegohaz )

@gziolo
Copy link
Member

gziolo commented Dec 14, 2021

Let's make sure it isn't related to the fact that @wordpress/components has outdated types for React. See more in #37365. There is also some sort of issue with types to resolve.

@youknowriad
Copy link
Contributor Author

The types are now fixed per @sirreal's suggestion, the e2e test are passing 🎉 still have some issues to solve for unit and mobile tests (probably missing dependencies)

@gziolo
Copy link
Member

gziolo commented Dec 15, 2021

I opened also #37396 to address some more issues discovered in this branch.

gziolo added a commit that referenced this pull request Dec 15, 2021
Props to @youknowriad to fixing all those subtle bug while exploring pnpm in Gutenberg with #37324.
@diegohaz
Copy link
Member

cc: @ciampo who may be able to check my work

Thank you for the ping Jon! I am not entirely sure if we need to exclude ref (cc @sarayourfriend ) and/or if reakit already exports the type for the Popover pros, which could potentially avoid calling ComponentPropsWithoutRef in the first place (cc @diegohaz )

Yeah! Reakit exports PopoverProps. But using ComponentProps or ComponentPropsWithoutRef should also be fine.

@sirreal
Copy link
Member

sirreal commented Dec 15, 2021

Let's make sure it isn't related to the fact that @wordpress/components has outdated types for React. See more in #37365. There is also some sort of issue with types to resolve.

👍 Agreed, it's worth seeing if the @types/react upgrade helps. I recall reaching v17 React types through Reakit, so it may indeed be related to those versions being out of sync. It's worth trying to revert the change I suggested after #37365 lands.

gziolo added a commit that referenced this pull request Dec 15, 2021
Props to @youknowriad to fixing all those subtle bug while exploring pnpm in Gutenberg with #37324.
gziolo added a commit that referenced this pull request Dec 15, 2021
Props to @youknowriad to fixing all those subtle bug while exploring pnpm in Gutenberg with #37324.
@gziolo gziolo added this to In progress in Core JS Dec 16, 2021
@github-actions
Copy link

github-actions bot commented Dec 29, 2021

Size Change: +607 B (0%)

Total Size: 1.13 MB

Filename Size Change
build/annotations/index.min.js 2.74 kB -7 B (0%)
build/blob/index.min.js 437 B -22 B (-5%)
build/block-directory/index.min.js 6.26 kB -15 B (0%)
build/block-editor/index.min.js 141 kB +695 B (0%)
build/block-library/index.min.js 164 kB -1.65 kB (-1%)
build/block-serialization-spec-parser/index.min.js 2.8 kB +2 B (0%)
build/blocks/index.min.js 47 kB +647 B (+1%)
build/components/index.min.js 217 kB +2.21 kB (+1%)
build/compose/index.min.js 11.1 kB -124 B (-1%)
build/core-data/index.min.js 13.1 kB -189 B (-1%)
build/customize-widgets/index.min.js 11.3 kB -82 B (-1%)
build/data/index.min.js 7.44 kB -46 B (-1%)
build/date/index.min.js 31.8 kB -49 B (0%)
build/dom/index.min.js 4.47 kB -31 B (-1%)
build/edit-navigation/index.min.js 15.8 kB -219 B (-1%)
build/edit-post/index.min.js 29.5 kB -106 B (0%)
build/edit-site/index.min.js 37.3 kB -403 B (-1%)
build/edit-widgets/index.min.js 16.4 kB -157 B (-1%)
build/editor/index.min.js 38.2 kB -206 B (-1%)
build/format-library/index.min.js 6.58 kB -9 B (0%)
build/hooks/index.min.js 1.61 kB -16 B (-1%)
build/i18n/index.min.js 3.79 kB +35 B (+1%)
build/keyboard-shortcuts/index.min.js 1.8 kB +2 B (0%)
build/media-utils/index.min.js 2.89 kB -28 B (-1%)
build/notices/index.min.js 939 B +14 B (+2%)
build/nux/index.min.js 2.09 kB +8 B (0%)
build/react-refresh-entry/index.min.js 8.95 kB +511 B (+6%) 🔍
build/reusable-blocks/index.min.js 2.21 kB -9 B (0%)
build/rich-text/index.min.js 11 kB -43 B (0%)
build/server-side-render/index.min.js 1.57 kB -6 B (0%)
build/viewport/index.min.js 1.04 kB -10 B (-1%)
build/widgets/index.min.js 7.1 kB -51 B (-1%)
build/wordcount/index.min.js 1 kB -36 B (-3%)
ℹ️ View Unchanged
Filename Size
build/a11y/index.min.js 960 B
build/admin-manifest/index.min.js 1.1 kB
build/api-fetch/index.min.js 2.21 kB
build/autop/index.min.js 2.12 kB
build/block-directory/style-rtl.css 1.01 kB
build/block-directory/style.css 1.01 kB
build/block-editor/default-editor-styles-rtl.css 378 B
build/block-editor/default-editor-styles.css 378 B
build/block-editor/style-rtl.css 14.6 kB
build/block-editor/style.css 14.6 kB
build/block-library/blocks/archives/editor-rtl.css 61 B
build/block-library/blocks/archives/editor.css 60 B
build/block-library/blocks/archives/style-rtl.css 65 B
build/block-library/blocks/archives/style.css 65 B
build/block-library/blocks/audio/editor-rtl.css 150 B
build/block-library/blocks/audio/editor.css 150 B
build/block-library/blocks/audio/style-rtl.css 111 B
build/block-library/blocks/audio/style.css 111 B
build/block-library/blocks/audio/theme-rtl.css 125 B
build/block-library/blocks/audio/theme.css 125 B
build/block-library/blocks/block/editor-rtl.css 161 B
build/block-library/blocks/block/editor.css 161 B
build/block-library/blocks/button/editor-rtl.css 470 B
build/block-library/blocks/button/editor.css 470 B
build/block-library/blocks/button/style-rtl.css 560 B
build/block-library/blocks/button/style.css 560 B
build/block-library/blocks/buttons/editor-rtl.css 292 B
build/block-library/blocks/buttons/editor.css 292 B
build/block-library/blocks/buttons/style-rtl.css 275 B
build/block-library/blocks/buttons/style.css 275 B
build/block-library/blocks/calendar/style-rtl.css 207 B
build/block-library/blocks/calendar/style.css 207 B
build/block-library/blocks/categories/editor-rtl.css 84 B
build/block-library/blocks/categories/editor.css 83 B
build/block-library/blocks/categories/style-rtl.css 79 B
build/block-library/blocks/categories/style.css 79 B
build/block-library/blocks/code/style-rtl.css 90 B
build/block-library/blocks/code/style.css 90 B
build/block-library/blocks/code/theme-rtl.css 131 B
build/block-library/blocks/code/theme.css 131 B
build/block-library/blocks/columns/editor-rtl.css 108 B
build/block-library/blocks/columns/editor.css 108 B
build/block-library/blocks/columns/style-rtl.css 406 B
build/block-library/blocks/columns/style.css 406 B
build/block-library/blocks/comment-template/style-rtl.css 127 B
build/block-library/blocks/comment-template/style.css 127 B
build/block-library/blocks/comments-pagination-numbers/editor-rtl.css 123 B
build/block-library/blocks/comments-pagination-numbers/editor.css 121 B
build/block-library/blocks/comments-pagination/editor-rtl.css 222 B
build/block-library/blocks/comments-pagination/editor.css 209 B
build/block-library/blocks/comments-pagination/style-rtl.css 235 B
build/block-library/blocks/comments-pagination/style.css 231 B
build/block-library/blocks/cover/editor-rtl.css 546 B
build/block-library/blocks/cover/editor.css 547 B
build/block-library/blocks/cover/style-rtl.css 1.22 kB
build/block-library/blocks/cover/style.css 1.22 kB
build/block-library/blocks/embed/editor-rtl.css 293 B
build/block-library/blocks/embed/editor.css 293 B
build/block-library/blocks/embed/style-rtl.css 417 B
build/block-library/blocks/embed/style.css 417 B
build/block-library/blocks/embed/theme-rtl.css 124 B
build/block-library/blocks/embed/theme.css 124 B
build/block-library/blocks/file/editor-rtl.css 300 B
build/block-library/blocks/file/editor.css 300 B
build/block-library/blocks/file/style-rtl.css 255 B
build/block-library/blocks/file/style.css 255 B
build/block-library/blocks/file/view.min.js 322 B
build/block-library/blocks/freeform/editor-rtl.css 2.44 kB
build/block-library/blocks/freeform/editor.css 2.44 kB
build/block-library/blocks/gallery/editor-rtl.css 965 B
build/block-library/blocks/gallery/editor.css 967 B
build/block-library/blocks/gallery/style-rtl.css 1.6 kB
build/block-library/blocks/gallery/style.css 1.6 kB
build/block-library/blocks/gallery/theme-rtl.css 122 B
build/block-library/blocks/gallery/theme.css 122 B
build/block-library/blocks/group/editor-rtl.css 159 B
build/block-library/blocks/group/editor.css 159 B
build/block-library/blocks/group/style-rtl.css 57 B
build/block-library/blocks/group/style.css 57 B
build/block-library/blocks/group/theme-rtl.css 78 B
build/block-library/blocks/group/theme.css 78 B
build/block-library/blocks/heading/style-rtl.css 114 B
build/block-library/blocks/heading/style.css 114 B
build/block-library/blocks/html/editor-rtl.css 332 B
build/block-library/blocks/html/editor.css 333 B
build/block-library/blocks/image/editor-rtl.css 810 B
build/block-library/blocks/image/editor.css 809 B
build/block-library/blocks/image/style-rtl.css 507 B
build/block-library/blocks/image/style.css 511 B
build/block-library/blocks/image/theme-rtl.css 124 B
build/block-library/blocks/image/theme.css 124 B
build/block-library/blocks/latest-comments/style-rtl.css 284 B
build/block-library/blocks/latest-comments/style.css 284 B
build/block-library/blocks/latest-posts/editor-rtl.css 199 B
build/block-library/blocks/latest-posts/editor.css 198 B
build/block-library/blocks/latest-posts/style-rtl.css 447 B
build/block-library/blocks/latest-posts/style.css 446 B
build/block-library/blocks/list/style-rtl.css 94 B
build/block-library/blocks/list/style.css 94 B
build/block-library/blocks/media-text/editor-rtl.css 266 B
build/block-library/blocks/media-text/editor.css 263 B
build/block-library/blocks/media-text/style-rtl.css 493 B
build/block-library/blocks/media-text/style.css 490 B
build/block-library/blocks/more/editor-rtl.css 431 B
build/block-library/blocks/more/editor.css 431 B
build/block-library/blocks/navigation-link/editor-rtl.css 649 B
build/block-library/blocks/navigation-link/editor.css 650 B
build/block-library/blocks/navigation-link/style-rtl.css 94 B
build/block-library/blocks/navigation-link/style.css 94 B
build/block-library/blocks/navigation-submenu/editor-rtl.css 299 B
build/block-library/blocks/navigation-submenu/editor.css 299 B
build/block-library/blocks/navigation-submenu/view.min.js 343 B
build/block-library/blocks/navigation/editor-rtl.css 1.99 kB
build/block-library/blocks/navigation/editor.css 2 kB
build/block-library/blocks/navigation/style-rtl.css 1.85 kB
build/block-library/blocks/navigation/style.css 1.84 kB
build/block-library/blocks/navigation/view.min.js 2.81 kB
build/block-library/blocks/nextpage/editor-rtl.css 395 B
build/block-library/blocks/nextpage/editor.css 395 B
build/block-library/blocks/page-list/editor-rtl.css 401 B
build/block-library/blocks/page-list/editor.css 402 B
build/block-library/blocks/page-list/style-rtl.css 175 B
build/block-library/blocks/page-list/style.css 175 B
build/block-library/blocks/paragraph/editor-rtl.css 157 B
build/block-library/blocks/paragraph/editor.css 157 B
build/block-library/blocks/paragraph/style-rtl.css 273 B
build/block-library/blocks/paragraph/style.css 273 B
build/block-library/blocks/post-author/style-rtl.css 175 B
build/block-library/blocks/post-author/style.css 176 B
build/block-library/blocks/post-comments-form/style-rtl.css 446 B
build/block-library/blocks/post-comments-form/style.css 446 B
build/block-library/blocks/post-comments/style-rtl.css 521 B
build/block-library/blocks/post-comments/style.css 521 B
build/block-library/blocks/post-excerpt/editor-rtl.css 73 B
build/block-library/blocks/post-excerpt/editor.css 73 B
build/block-library/blocks/post-excerpt/style-rtl.css 69 B
build/block-library/blocks/post-excerpt/style.css 69 B
build/block-library/blocks/post-featured-image/editor-rtl.css 721 B
build/block-library/blocks/post-featured-image/editor.css 721 B
build/block-library/blocks/post-featured-image/style-rtl.css 153 B
build/block-library/blocks/post-featured-image/style.css 153 B
build/block-library/blocks/post-template/editor-rtl.css 99 B
build/block-library/blocks/post-template/editor.css 98 B
build/block-library/blocks/post-template/style-rtl.css 323 B
build/block-library/blocks/post-template/style.css 323 B
build/block-library/blocks/post-terms/style-rtl.css 73 B
build/block-library/blocks/post-terms/style.css 73 B
build/block-library/blocks/post-title/style-rtl.css 80 B
build/block-library/blocks/post-title/style.css 80 B
build/block-library/blocks/preformatted/style-rtl.css 103 B
build/block-library/blocks/preformatted/style.css 103 B
build/block-library/blocks/pullquote/editor-rtl.css 198 B
build/block-library/blocks/pullquote/editor.css 198 B
build/block-library/blocks/pullquote/style-rtl.css 389 B
build/block-library/blocks/pullquote/style.css 388 B
build/block-library/blocks/pullquote/theme-rtl.css 167 B
build/block-library/blocks/pullquote/theme.css 167 B
build/block-library/blocks/query-pagination-numbers/editor-rtl.css 122 B
build/block-library/blocks/query-pagination-numbers/editor.css 121 B
build/block-library/blocks/query-pagination/editor-rtl.css 221 B
build/block-library/blocks/query-pagination/editor.css 211 B
build/block-library/blocks/query-pagination/style-rtl.css 234 B
build/block-library/blocks/query-pagination/style.css 231 B
build/block-library/blocks/query/editor-rtl.css 131 B
build/block-library/blocks/query/editor.css 132 B
build/block-library/blocks/quote/style-rtl.css 187 B
build/block-library/blocks/quote/style.css 187 B
build/block-library/blocks/quote/theme-rtl.css 223 B
build/block-library/blocks/quote/theme.css 226 B
build/block-library/blocks/rss/editor-rtl.css 202 B
build/block-library/blocks/rss/editor.css 204 B
build/block-library/blocks/rss/style-rtl.css 289 B
build/block-library/blocks/rss/style.css 288 B
build/block-library/blocks/search/editor-rtl.css 165 B
build/block-library/blocks/search/editor.css 165 B
build/block-library/blocks/search/style-rtl.css 397 B
build/block-library/blocks/search/style.css 398 B
build/block-library/blocks/search/theme-rtl.css 64 B
build/block-library/blocks/search/theme.css 64 B
build/block-library/blocks/separator/editor-rtl.css 99 B
build/block-library/blocks/separator/editor.css 99 B
build/block-library/blocks/separator/style-rtl.css 245 B
build/block-library/blocks/separator/style.css 245 B
build/block-library/blocks/separator/theme-rtl.css 172 B
build/block-library/blocks/separator/theme.css 172 B
build/block-library/blocks/shortcode/editor-rtl.css 474 B
build/block-library/blocks/shortcode/editor.css 474 B
build/block-library/blocks/site-logo/editor-rtl.css 744 B
build/block-library/blocks/site-logo/editor.css 744 B
build/block-library/blocks/site-logo/style-rtl.css 181 B
build/block-library/blocks/site-logo/style.css 181 B
build/block-library/blocks/site-tagline/editor-rtl.css 86 B
build/block-library/blocks/site-tagline/editor.css 86 B
build/block-library/blocks/site-title/editor-rtl.css 84 B
build/block-library/blocks/site-title/editor.css 84 B
build/block-library/blocks/social-link/editor-rtl.css 177 B
build/block-library/blocks/social-link/editor.css 177 B
build/block-library/blocks/social-links/editor-rtl.css 674 B
build/block-library/blocks/social-links/editor.css 673 B
build/block-library/blocks/social-links/style-rtl.css 1.32 kB
build/block-library/blocks/social-links/style.css 1.32 kB
build/block-library/blocks/spacer/editor-rtl.css 332 B
build/block-library/blocks/spacer/editor.css 332 B
build/block-library/blocks/spacer/style-rtl.css 48 B
build/block-library/blocks/spacer/style.css 48 B
build/block-library/blocks/table/editor-rtl.css 471 B
build/block-library/blocks/table/editor.css 472 B
build/block-library/blocks/table/style-rtl.css 481 B
build/block-library/blocks/table/style.css 481 B
build/block-library/blocks/table/theme-rtl.css 188 B
build/block-library/blocks/table/theme.css 188 B
build/block-library/blocks/tag-cloud/style-rtl.css 214 B
build/block-library/blocks/tag-cloud/style.css 215 B
build/block-library/blocks/template-part/editor-rtl.css 560 B
build/block-library/blocks/template-part/editor.css 559 B
build/block-library/blocks/template-part/theme-rtl.css 101 B
build/block-library/blocks/template-part/theme.css 101 B
build/block-library/blocks/text-columns/editor-rtl.css 95 B
build/block-library/blocks/text-columns/editor.css 95 B
build/block-library/blocks/text-columns/style-rtl.css 166 B
build/block-library/blocks/text-columns/style.css 166 B
build/block-library/blocks/verse/style-rtl.css 87 B
build/block-library/blocks/verse/style.css 87 B
build/block-library/blocks/video/editor-rtl.css 571 B
build/block-library/blocks/video/editor.css 572 B
build/block-library/blocks/video/style-rtl.css 173 B
build/block-library/blocks/video/style.css 173 B
build/block-library/blocks/video/theme-rtl.css 124 B
build/block-library/blocks/video/theme.css 124 B
build/block-library/common-rtl.css 908 B
build/block-library/common.css 905 B
build/block-library/editor-rtl.css 10.1 kB
build/block-library/editor.css 10.1 kB
build/block-library/reset-rtl.css 474 B
build/block-library/reset.css 474 B
build/block-library/style-rtl.css 10.8 kB
build/block-library/style.css 10.8 kB
build/block-library/theme-rtl.css 672 B
build/block-library/theme.css 676 B
build/block-serialization-default-parser/index.min.js 1.09 kB
build/components/style-rtl.css 15.5 kB
build/components/style.css 15.5 kB
build/customize-widgets/style-rtl.css 1.5 kB
build/customize-widgets/style.css 1.49 kB
build/data-controls/index.min.js 631 B
build/deprecated/index.min.js 485 B
build/dom-ready/index.min.js 304 B
build/edit-navigation/style-rtl.css 3.76 kB
build/edit-navigation/style.css 3.76 kB
build/edit-post/classic-rtl.css 546 B
build/edit-post/classic.css 547 B
build/edit-post/style-rtl.css 7.15 kB
build/edit-post/style.css 7.14 kB
build/edit-site/style-rtl.css 6.85 kB
build/edit-site/style.css 6.84 kB
build/edit-widgets/style-rtl.css 4.17 kB
build/edit-widgets/style.css 4.17 kB
build/editor/style-rtl.css 3.71 kB
build/editor/style.css 3.71 kB
build/element/index.min.js 3.29 kB
build/escape-html/index.min.js 517 B
build/format-library/style-rtl.css 571 B
build/format-library/style.css 571 B
build/html-entities/index.min.js 424 B
build/is-shallow-equal/index.min.js 501 B
build/keycodes/index.min.js 1.39 kB
build/list-reusable-blocks/index.min.js 1.72 kB
build/list-reusable-blocks/style-rtl.css 838 B
build/list-reusable-blocks/style.css 838 B
build/nux/style-rtl.css 747 B
build/nux/style.css 743 B
build/plugins/index.min.js 1.84 kB
build/primitives/index.min.js 924 B
build/priority-queue/index.min.js 582 B
build/react-i18n/index.min.js 671 B
build/react-refresh-runtime/index.min.js 7.31 kB
build/redux-routine/index.min.js 2.65 kB
build/reusable-blocks/style-rtl.css 256 B
build/reusable-blocks/style.css 256 B
build/shortcode/index.min.js 1.49 kB
build/token-list/index.min.js 639 B
build/url/index.min.js 1.9 kB
build/warning/index.min.js 248 B
build/widgets/style-rtl.css 1.16 kB
build/widgets/style.css 1.16 kB

compressed-size-action

@kevin940726
Copy link
Member

I just discovered this and I just want to say that I would very much like this to happen. Could we share a before&after of the installation time and disk size comparison in the PR description?

One thing that might be worth doing is to only allow pnpm, so that people won't accidentally install a package using npm install and wait for several minutes just to find out that it's the wrong command.

@youknowriad
Copy link
Contributor Author

Just wanted to share that I'm not working on this anymore, anyone can feel free to take it over. I think it's valuable to update to use pnpm (or else npm latest)

@sarayourfriend
Copy link
Contributor

sarayourfriend commented Jan 21, 2022

I'll take a look at this today and see if I can make any extra headway. We've benefitted a lot from using pnpm with Openverse.

I'm not sure how these work, so I will flag them explicitly
for review. I suspect the difference just comes from how
pnpm is resolving dependencies vs how npm was doing it, but
just in case this actually indicates some erroneous dependency
resolution we should double check it.
@gziolo
Copy link
Member

gziolo commented Jan 21, 2022

I'll take a look at this today and see if I can make any extra headway. We've benefitted a lot from using pnpm with Openverse.

Can you elaborate on the benefits? How does it compare to npm 8? Do you hoist all packages to the top of the project?

@sarayourfriend
Copy link
Contributor

Can you elaborate on the benefits? How does it compare to npm 8? Do you hoist all packages to the top of the project?

We were on npm 8 before pnpm. The main benefit is super fast installs locally when dependencies change. The first time I installed the Gutenberg dependencies with pnpm today only a few hundred new dependecies were downloaded, the rest already existed in my local cache, and symlinking is super fast. This also makes debugging different setups (like hoisting) much faster. This benefit is also translated to CI when using a dependency cache.

Switching to pnpm also fixed a whole host of dependency resolution issues we were having with Storybook (something I know Gutenberg has also struggled with in the past).

The PR where Openverse switched to pnpm is here: WordPress/openverse-frontend#525

We do hoist. Vue requires this for now (I believe Vue 3 and the latest Nuxt versions might fix this but we're stuck on Nuxt 2 and Vue 2 for now until the Nuxt upgrade path is more sensible and stable).

We also had to enable enable-pre-post-scripts, also for Vue support. Gutenberg also requires this option to be enabled as far as I can tell.

A significant difference for Openverse is that our repository is not a monorepo with workspaces. The openverse-frontend repository follows the normal Nuxt structure of a monolith app with no packages folder (or any equivalent). We don't currently have any reason to do this (maybe in the future we will if our components become more general purpose or something). We've explored moving to a monorepo briefly but the rest of our stack is Python and there are significant issues with managing Python monorepos (even our openverse-api repository is something of a Python monorepo and it's a pain to get editors to pick up the correct dependencies for each project in the repository due to how Pipenv functions).

I'll need to do some reading into how pnpm wants to manage workspaces and make sure that there's nothing specifically different that needs to be done.

Meanwhile, enabling pre and post scripts fixes a lot of the errors seen in the CI on the latest commit of this branch as of writing this comment.

@gziolo
Copy link
Member

gziolo commented Jan 22, 2022

@sarayourfriend, thank you for a detailed response. I will process it early next week 👍🏻

I had a branch ready #36041 with changes for npm 8, but it doesn't seem to install correctly with GitHub Actions. It worked locally without any issues. So you can't compare install times using CI, for now. 😅 We could still be able to run some comparisons locally.

@gziolo
Copy link
Member

gziolo commented Jan 24, 2022

The PR where Openverse switched to pnpm is here: WordPress/openverse-frontend#525

It feels like the biggest challenge would be to update all commands from npm run to pnpm alternatives in the Gutenberg codebase. In particular, it's tricky with WordPress packages where we want to keep npm run for simplicity. The same applies to documentation targeting folks developing on top of WordPress. So this all boils down to identifying those places where we should teach Gutenberg contributors how to use pnpm. It's also important to figure out how to integrate a good check that ensures that all contributors start using pnpm and more importantly they see warnings on the terminal that they should switch to pnpm from npm.

I took a look at CI jobs like https://github.com/WordPress/gutenberg/runs/4898743594?check_suite_focus=true that run pnpm install and it doesn't seem to be significantly faster than npm ci with npm 6. I can't compare it with npm 8 because it doesn't install correctly as I mentioned in my previous comment.

@sarayourfriend
Copy link
Contributor

sarayourfriend commented Jan 24, 2022

I took a look at CI jobs like https://github.com/WordPress/gutenberg/runs/4898743594?check_suite_focus=true that run pnpm install and it doesn't seem to be significantly faster than npm ci with npm 6. I can't compare it with npm 8 because it doesn't install correctly as I mentioned in my previous comment.

To be fair, the CI action you're looking at there is running cachelessly as there's a brand new pnpm-lock.yaml. I also think the cache strategy used by Gutenberg might not work for pnpm, but I'm not sure. We use a different approach in Openverse to use cached node_modules between installs.

I think that a "raw, from-zero" install using pnpm isn't going to be faster than npm as the main bottle neck is the network, not putting the files into node_modules. pnpm definitely shines for local installs where your local cache of packages is going to be significant if you've done development on any other moderately sized JavaScript projects. I think it could also speed up CI if the correct caching is used.

For the record, this is the caching we use for Openverse: https://github.com/marketplace/actions/setup-pnpm#use-cache-to-reduce-installation-time. We just follow the official recommendation for pnpm.

@sarayourfriend
Copy link
Contributor

Taking another look at this PR today. Going to try to start from fresh and make more incremental changes. Obviously some of the changes I made broke things more than helped! I've also learned since that React Native doesn't play well necessarily with pnpm and requires some manual steps, which I was getting tripped up on.

Most likely I will start a new branch and run a fresh pnpm import and then take things step by step.

@gziolo
Copy link
Member

gziolo commented Feb 8, 2022

I've also learned since that React Native doesn't play well necessarily with pnpm and requires some manual steps, which I was getting tripped up on.

There are also issues with React Native forked dependencies and npm 8 on CI. It works correctly locally.

@sarayourfriend sarayourfriend mentioned this pull request Feb 8, 2022
8 tasks
@sarayourfriend
Copy link
Contributor

I made a lot of progress today in #38624.

It's far from perfect and there are blockers (compressed-size-check doesn't support pnpm yet) but I got a lot closer today in that branch than I did on this PR. I am hopeful that there is a way to get everything passing except the compressed size check and the PR automation.

Core JS automation moved this from In progress to Done Sep 19, 2022
@youknowriad youknowriad deleted the try/pnpm branch September 19, 2022 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Build Tooling Issues or PRs related to build tooling
Projects
No open projects
Core JS
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

7 participants