Reduce the number of npm dependencies needed by parcel #7576

devongovett · 2022-01-19T17:33:11Z

Previously, installing the parcel package downloaded 493 dependencies from npm. This was made better from the over 600 deps we used to have in #7564, but is still pretty ridiculous. The aim of this PR is to reduce this further. This is accomplished in a few ways:

Pre-bundling some dependencies with Parcel itself rather than loading them from npm. This is done mainly for small utility dependencies, not for things that applications would normally need to control the versions of. Specifically, for the following packages: @parcel/fs, @parcel/package-manager, @parcel/utils, @parcel/codeframe, @parcel/reporter-cli, and @parcel/reporter-dev-server. This reduces dependencies to 334. It also has the benefit that any random security warnings produced by npm will only affect the Parcel team and not downstream users. Most of the security warnings we receive are not actually exploitable in Parcel anyway, so this really is just about reducing annoyance for everyone. Both Next.js and Vite have taken a similar approach.
Auto installing node builtin polyfills on demand (e.g. buffer, stream, etc.). The @parcel/node-libs-browser package pulled in over 100 dependencies, which were pretty rarely used these days. Now you'll see a warning when one is encountered, and it'll be auto installed into your project. In combination with the above, this reduces dependencies to 226.
Removing built-in Babel and PostCSS dependencies, and installing them into projects on demand, only when actually used. This includes builtin flow and legacy css modules support. In combination with the above, this reduces dependencies to 183.
Removing CSS nano once we switch entirely to @parcel/css will reduce this further to 150.

There are other potential optimizations we can do in the future, but this already helps a lot, reducing dependencies by ~70%. It should result in faster install times, and less dependencies to audit for everyone using Parcel.

height · 2022-01-19T17:33:12Z

Link Height tasks by mentioning a task ID in the pull request title or commit messages, or description and comments with the keyword link (e.g. "Link T-123").

💡Tip: You can also use "Close T-X" to automatically close a task when the pull request is merged.

devongovett · 2022-01-19T17:35:10Z

packages/core/core/src/ParcelConfig.js

@@ -29,7 +29,7 @@ import ThrowableDiagnostic, {
 } from '@parcel/diagnostic';
 import json5 from 'json5';

-import {makeRe} from 'micromatch';
+import {globToRegex} from '@parcel/utils';


Centralized all uses of micromatch into the @parcel/utils package so we only get one copy

I recall that there was some breaking change in micromatch that thwarted a previous attempt to consolidate it (see this comment from @mischnic on a PR I forgot about 😨 ). Not sure if it's still relevant?

At least one occurence was fixed in #6958

devongovett · 2022-01-19T17:36:13Z

packages/core/parcel/src/bin.js

-if (process.env.PARCEL_BUILD_ENV !== 'production') {
+if (
+  process.env.PARCEL_BUILD_ENV !== 'production' ||
+  process.env.PARCEL_SELF_BUILD


This new environment variable is needed so we can replace PARCEL_BUILD_ENV while parcel is building itself...

devongovett · 2022-01-19T17:36:53Z

packages/dev/babel-preset/index.js

-    production: {
+module.exports = api => {
+  let name = api.caller(caller => caller && caller.name);
+  if (name === 'parcel') {


A different Babel config is used when Parcel is building itself since we don't want to compile modules etc.

devongovett · 2022-01-19T17:38:33Z

packages/transformers/postcss/src/PostCSSTransformer.js

@@ -79,10 +77,22 @@ export default (new Transformer({
    let cssModules: ?{|[string]: string|} = null;
    if (config.hydrated.modules) {
      asset.meta.cssModulesCompiled = true;
+
+      // TODO: should this be resolved from the project root?


actually a question for all dev deps. What if there were a CSS module in node_modules? we would never want to install dev deps there...

devongovett · 2022-01-19T17:39:45Z

packages/utils/node-resolver-core/src/NodeResolver.js

@@ -602,7 +653,16 @@ export default class NodeResolver {
      if (isExplicitNode) {
        filename = filename.substr(5);
      }
-      return {filePath: builtins[filename] || empty};
+
+      // By default, exclude node builtins from libraries unless explicitly opted in.


Is this a good idea? Alternatively we could have some separate config for it, but it seems like when building libraries you'd usually want to exclude these...

Since it is ideal to resolve node builtins polyfill with the bundler of the library user, I feel we do not want to include node builtins if a bundling target is a library working in the browser. On the other hand, I think some library author wants to host their library on esm.sh etc. and use it as pure ESModules from the browser. In that case, the bundle of the library should include node builtins.

So, I think it is good to make it the default behavior, but I think also that an emergency hatch should be prepared as a config or flag for such use cases. (it would be simple to judge that with whether or not the output is esmodule, but in reality, it shouldn't be so difficult ….)

I'm sorry if I overlooked the context and said something strange

packages/utils/node-resolver-core/src/NodeResolver.js

mischnic · 2022-01-20T20:41:14Z

packages/utils/node-resolver-core/test/resolver.js

@@ -297,8 +297,8 @@ describe('resolver', function () {
            aboveFilePath: path.join(rootDir, 'foo.js'),
          },
          {
-            fileName: 'package.json',
-            aboveFilePath: require.resolve('browserify-zlib'),
+            fileName: 'node_modules/browserify-zlib',


I'm surprised that invalidateOnFileCreate can be used to watch for folder creations...

Missed in #7576

xvello · 2022-02-05T15:13:13Z

Thanks for this @devongovett, do you have plans to cut a release including these changes soon?

devongovett · 2022-02-05T15:23:07Z

@xvello i hope so. We just want to do some testing to ensure nothing is broken. If you want, you could try out the nightly release and report back if it works for you?

xvello · 2022-02-05T16:02:49Z

@devongovett I tried upgrading my simple project to the nightlies, and the generated output is the same, so good enough for me I guess? ;)

npm install --dev parcel@2.0.0-nightly.991
npm install -dev @parcel/transformer-sass@2.0.0-nightly.993+90050e4e

I reached out to know whether I should wait for an upcoming release, or commit these updates.

devongovett added 12 commits January 17, 2022 19:59

Pre-bundle some dependencies

af0fc13

HACK: allow excluding node builtins

6c5db4f

Support aliases of excluded dependencies

25bc652

Remove babel dependencies by default

dce7a8c

Lazy install postcss-modules

780278b

Compile ws

22775b6

Load postcss from project in CSS packager

c688065

Install node builtin polyfills on demand

d59b67f

Bug fixes

4265e96

Fix flaky test

4bbb6a1

cleanup

33e6e7e

Make build work

8dd2ff4

devongovett commented Jan 19, 2022

View reviewed changes

Fix lint

9a27670

mischnic reviewed Jan 20, 2022

View reviewed changes

packages/utils/node-resolver-core/src/NodeResolver.js Show resolved Hide resolved

mischnic reviewed Jan 20, 2022

View reviewed changes

Fix autoinstall issue

c03f525

devongovett merged commit 4cbda87 into v2 Jan 27, 2022

devongovett deleted the reduce-deps branch January 27, 2022 15:47

devongovett added a commit that referenced this pull request Jan 31, 2022

Remove postcss-modules dependency by default

5b38bdf

Missed in #7576

azu mentioned this pull request Feb 14, 2022

2022-02-14のJS: Parcel 2.3.0、Vite 2.8.0、Angular Compilerがどのように動いているか、JavaScriptでDDD jser/jser.github.io#959

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce the number of npm dependencies needed by parcel #7576

Reduce the number of npm dependencies needed by parcel #7576

devongovett commented Jan 19, 2022 •

edited

height bot commented Jan 19, 2022

devongovett Jan 19, 2022

lettertwo Jan 19, 2022

mischnic Jan 20, 2022

devongovett Jan 19, 2022

devongovett Jan 19, 2022

devongovett Jan 19, 2022

devongovett Jan 19, 2022

Shinyaigeek Jan 20, 2022 •

edited

mischnic Jan 20, 2022 •

edited

xvello commented Feb 5, 2022

devongovett commented Feb 5, 2022

xvello commented Feb 5, 2022 •

edited

Reduce the number of npm dependencies needed by parcel #7576

Reduce the number of npm dependencies needed by parcel #7576

Conversation

devongovett commented Jan 19, 2022 • edited

height bot commented Jan 19, 2022

devongovett Jan 19, 2022

Choose a reason for hiding this comment

lettertwo Jan 19, 2022

Choose a reason for hiding this comment

mischnic Jan 20, 2022

Choose a reason for hiding this comment

devongovett Jan 19, 2022

Choose a reason for hiding this comment

devongovett Jan 19, 2022

Choose a reason for hiding this comment

devongovett Jan 19, 2022

Choose a reason for hiding this comment

devongovett Jan 19, 2022

Choose a reason for hiding this comment

Shinyaigeek Jan 20, 2022 • edited

Choose a reason for hiding this comment

mischnic Jan 20, 2022 • edited

Choose a reason for hiding this comment

xvello commented Feb 5, 2022

devongovett commented Feb 5, 2022

xvello commented Feb 5, 2022 • edited

devongovett commented Jan 19, 2022 •

edited

Shinyaigeek Jan 20, 2022 •

edited

mischnic Jan 20, 2022 •

edited

xvello commented Feb 5, 2022 •

edited