Skip to content
This repository has been archived by the owner on Sep 2, 2023. It is now read-only.

Latest commit

 

History

History
146 lines (101 loc) · 11.4 KB

plan-for-new-modules-implementation.md

File metadata and controls

146 lines (101 loc) · 11.4 KB
Raw

Plan for New Modules Implementation

This document outlines the plan for building a new implementation to support ECMAScript modules in Node.js. The effort is split up into phases:

  • Phase 0 branches off of current Node but removes much of the Node 8.5.0+ --experimental-modules implementation so that a new implementation could be built in its place.

  • Phase 1 adds the “minimal kernel,” features that the modules working group felt would likely appear in any potential new ES modules implementation.

  • Phase 2 fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product.

    • At the completion of Phase 2, the old --experimental-modules implementation was replaced with this new one (still behind the --experimental-modules flag). It was released as part of Node 12 on 2019-04-23.

  • Phase 3 improves user experience and extends the MVP.

    • At the completion of Phase 3, the new implementation’s experimental flag will be dropped. The goal is to “release” (drop the --experimental-modules flag) by when Node 12 starts LTS in October 2019.

  • Phase 4 are items that were under development in earlier phases but weren’t finished when the new implementation’s experimental flag was dropped; these items may continue development after unflagging and potentially ship in later versions of Node.js.

The effort is currently in Phase 3.

At every phase, the following standards must be maintained:

  • Spec compliance (#132): We must always follow the ES spec.
  • Browser equivalence (#133): There’s room for debate in specific cases, but in general if Node is doing something that browsers also do, Node should do it in the same way. Alternatively, code that executes in both environments should produce identical results.
  • Don’t break CommonJS (#112): We cannot cause breaking changes with regards to CommonJS.

See also the features list in the README.

Phase 0: Start Fresh

From current shipping Node, the following changes were made to strip out most of the Node 8.5.0+ --experimental-modules implementation so that a new implementation could be built in its place:

  • Remove support in the import statement of formats other than ESM:

    • No CommonJS.
    • No JSON.
    • No native modules.

  • Remove dynamic path searching:

    • No extension adding.
    • No directory resolution, including no support for index.js or index.mjs.
    • No support for main field for ESM.

  • Remove current VM implementation

  • Remove current Loader implementation

These changes were implemented in nodejs/ecmascript-modules#6.

Phase 1: The Minimal Kernel

The “minimal kernel” consists of features that the @nodejs/modules group have agreed will be necessary for all potential iterations of our ESM implementation. Phase 1 does not include features that preclude other potential features or implementation approaches; and Phase 1 also does not include some features that should naturally be built in a later phase of development, for example because those features depend on features planned for Phase 1.

  • module.createRequireFromPath (nodejs/node#19360) is the only way to import CommonJS into an ES module, for now.

  • import statements will only support files with an .mjs extension, and will import only ES modules, for now.

    • No JSON or native modules; createRequireFromPath can be used to get these.

  • import.meta.url.

    • Already in the existing implementation.

  • Dynamic import().

    • Already in the existing implementation.

  • Support for built-in modules with named exports

    • Already in the existing implementation.

Phase 2: Minimum Viable Product: Required to Upstream

Phase 2 fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product. At the completion of Phase 2, the old --experimental-modules implementation was replaced with this new one (still behind the --experimental-modules flag).

  • Define semantics for importing a package entry point, e.g. import _ from 'lodash'

  • Define semantics for determining when to load sources as CommonJS or ES module for both the top-level main (node x.js) and dependency loading.

  • Define semantics for enabling ESM treatment of source code loaded via --eval, STDIN, and extensionless files (both with and without shebang lines).

    • Proposal: “Entry Points Proposal” covers non-file forms of input as well as adding --type flag for controlling file-based input.
    • Landed in nodejs/ecmascript-modules#32.
    • Renamed to --entry-type as part of upstream PR to Node.js core.
    • Renamed to --intry-type and limited to --eval, --print and STDIN as part of follow-up PR to Node.js core.

  • File extension and directory index searching in ESM, behind its own flag, --es-module-specifier-resolution.

The work through the end of Phase 2 landed in Node.js master as part of nodejs/node#26745 and was released in Node 12.0.0.

Phase 3: Path to Stability: Removing --experimental-modules Flag

Phase 3 improves user experience and extends the MVP. Phase 3 is malleable based on how things proceed while working on this phase. At the end of this phase, the --experimental-modules flag is dropped.

  • Better mechanism for creating require function: createRequire.

  • "exports" field: for consumers of a package, map the paths of deep imports to provide encapsulation (an explicit public API); pretty specifiers (no file exensions or paths that include things like dist/) and flexibility for future package versions renaming or moving files without affecting the package’s public API. Applies to both ESM and CommonJS.

    • Proposal: Package Exports Proposal.
    • Landed in nodejs/node#28568 and shipped in 12.7.0 behind --experimental-exports. Further improvements are being made as additional PRs against core.
    • The separate --experimental-exports flag was dropped in nodejs/node#29867, merging the feature with overall --experimental-modules.

  • Provide a way to make ESM the default instead of CommonJS.

    • Discussion: #318.
    • Proposal: #335.
    • Tabled. Currently one can add --input-type=module to NODE_OPTIONS to flip the default for --input-type; at the moment the group is deciding not to pursue providing an ability to flip the default for the package.json type field. We instead want to encourage all packages, both ESM and CommonJS, to include an explicit type field; leaving it out and relying on a default is something we want to discourage.

  • Define behavior for builtin globals between CommonJS and ESM. Does modifying a builtin in one module system carry over into the other? If it does, we may have major performance concerns.

    • Issue: nodejs/node#29426.
    • Current plan is to remove proxy support for changes in one system to affect the other.

Phase 4: Further Improvements After Unflagging

  • A loaders solution that supports all items in the features list in our README.

  • Shortcut to resolve to package root.

  • Limited module type detection.

  • Dual CommonJS/ESM packages: Either support packages with both CommonJS and ESM sources that can be used in either environment; or decide to specifically not support dual CommonJS/ESM packages.

    • Status quo (at time of possible unflagging): "main" points to exactly one file, and all file extensions are mandatory (by default), so there is no possibility of an import specifier pointing to different files in ESM versus CommonJS. Recommended practice for dual packages is to have "main" point to the CommonJS entry point and have users use a deep import, e.g. /module.mjs, to access ESM entry point.
    • Proposal for new package.json field for ESM entry point: #273.
    • Proposal for require of ESM: https://github.com/nodejs/modules/issues/299.
    • Status: “New field” proposal is blocked. “require of ESM” proposal may ship after unflagging if it is implemented and finds approval upstream.

  • Finalize behavior of import of CommonJS files and packages.

    • Overview: #264.
    • Status quo (at time of possible unflagging): import only the CommonJS default export, so import _ from 'cjs-pkg' but not import { shuffle } from 'cjs-pkg'.
    • If the spec changes to allow it, we want to implement dynamic modules to enable named exports from CommonJS: #252.
    • Another option is to specify CommonJS named exports in package.json: #324.
    • Status: Dynamic modules is stalled due to upstream concerns from TC39. Named exports in package.json seeks consensus but could potentially ship after unflagging.