From c469d4ba5a4eb6ef94bdaefa84f5e01dbcfce0bb Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Sat, 9 Nov 2019 11:54:13 -0800 Subject: [PATCH 1/9] doc: esm: improve dual package hazard docs --- doc/api/esm.md | 85 +++++++++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 36 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index f36470632b4763..3378cf36166dae 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -423,9 +423,6 @@ thrown: ### Dual CommonJS/ES Module Packages -_These patterns are currently experimental and only work under the -`--experimental-conditional-exports` flag._ - Prior to the introduction of support for ES modules in Node.js, it was a common pattern for package authors to include both CommonJS and ES module JavaScript sources in their package, with `package.json` `"main"` specifying the CommonJS @@ -434,14 +431,15 @@ This enabled Node.js to run the CommonJS entry point while build tools such as bundlers used the ES module entry point, since Node.js ignored (and still ignores) the top-level `"module"` field. -Node.js can now run ES module entry points, and using [Conditional Exports][] -with the `--experimental-conditional-exports` flag it is possible to define -separate package entry points for CommonJS and ES module consumers. Unlike in -the scenario where `"module"` is only used by bundlers, or ES module files are +Node.js can now run ES module entry points, and a package can contain both +CommonJS and ES module entry points (either via separate specifiers such as +`'pkg'` and `'pkg/module'`, or both at the same specifier via [Conditional +Exports][] with the `--experimental-conditional-exports` flag). Unlike in the +scenario where `"module"` is only used by bundlers, or ES module files are transpiled into CommonJS on the fly before evaluation by Node.js, the files referenced by the ES module entry point are evaluated as ES modules. -#### Divergent Specifier Hazard +#### Dual Package Hazard When an application is using a package that provides both CommonJS and ES module sources, there is a risk of certain bugs if both versions of the package get @@ -456,39 +454,38 @@ package might look like this: "type": "module", "main": "./pkg.cjs", "exports": { - "require": "./pkg.cjs", - "default": "./pkg.mjs" + ".": "./pkg.cjs", + "./module": "./pkg.mjs" } } ``` In this example, `require('pkg')` always resolves to `pkg.cjs`, including in versions of Node.js where ES modules are unsupported. In Node.js where ES -modules are supported, `import 'pkg'` references `pkg.mjs`. +modules are supported, `import 'pkg/module'` references `pkg.mjs`. The potential for bugs comes from the fact that the `pkg` created by `const pkg = require('pkg')` is not the same as the `pkg` created by `import pkg from -'pkg'`. This is the “divergent specifier hazard,” where one specifer (`'pkg'`) -resolves to separate files (`pkg.cjs` and `pkg.mjs`) in separate module systems, -yet both versions might get loaded within an application because Node.js -supports intermixing CommonJS and ES modules. - -If the export is a constructor, an `instanceof` comparison of instances created -by the two returns `false`, and if the export is an object, properties added to -one (like `pkg.foo = 3`) are not present on the other. This differs from how -`import` and `require` statements work in all-CommonJS or all-ES module -environments, respectively, and therefore is surprising to users. It also -differs from the behavior users are familiar with when using transpilation via -tools like [Babel][] or [`esm`][]. - -Even if the user consistently uses either `require` or `import` to refer to -`pkg`, if any dependencies of the application use the other method the hazard is -still present. - -The `--experimental-conditional-exports` flag should be set for modern Node.js -for this behavior to work out. If it is not set, only the ES module version can -be used in modern Node.js and the package will throw when accessed via -`require()`. +'pkg/module'`. This is the “dual package hazard,” where two versions of the same +package can be loaded within the same runtime environment. While it is unlikely +that an application or package would intentionally load both versions directly, +it is common for an application to load one version while a dependency of the +application loads the other version. This hazard can happen because Node.js +supports intermixing CommonJS and ES modules, and can lead to unexpected +behavior. + +The hazard is also present when [Conditional Exports][] with the +`--experimental-conditional-exports` flag are used. In that case, instead of +`'pkg'` and `'pkg/module'` as in the example above, both `require` and `import` +would use `'pkg'`. The hazard is the same. + +If the package main export is a constructor, an `instanceof` comparison of +instances created by the two versions returns `false`, and if the export is an +object, properties added to one (like `pkg.foo = 3`) are not present on the +other. This differs from how `import` and `require` statements work in +all-CommonJS or all-ES module environments, respectively, and therefore is +surprising to users. It also differs from the behavior users are familiar with +when using transpilation via tools like [Babel][] or [`esm`][]. #### Writing Dual Packages While Avoiding or Minimizing Hazards @@ -524,8 +521,9 @@ following conditions: Write the package in CommonJS or transpile ES module sources into CommonJS, and create an ES module wrapper file that defines the named exports. Using -[Conditional Exports][], the ES module wrapper is used for `import` and the -CommonJS entry point for `require`. +[Conditional Exports][] via the `--experimental-conditional-exports` flag, the +ES module wrapper is used for `import` and the CommonJS entry point for +`require`. ```js @@ -586,12 +584,27 @@ an all-ES module-syntax version the package. This could be used via `import 'pkg/module'` by users who are certain that the CommonJS version will not be loaded anywhere in the application, such as by dependencies; or if the CommonJS version can be loaded but doesn’t affect the ES module version (for example, -because the package is stateless). +because the package is stateless). This variant would not require +`--experimental-conditional-exports`: + + +```js +// ./node_modules/pkg/package.json +{ + "type": "module", + "main": "./index.cjs", + "exports": { + ".": "./index.cjs", + "./module": "./wrapper.mjs" + } +} +``` ##### Approach #2: Isolate State The most straightforward `package.json` would be one that defines the separate -CommonJS and ES module entry points directly: +CommonJS and ES module entry points directly (requires +`--experimental-conditional-exports`): ```js From 652613068d806cbd66321ed3feb3884a213fd139 Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Mon, 11 Nov 2019 00:12:42 -0800 Subject: [PATCH 2/9] doc: esm: remove example from hazard section --- doc/api/esm.md | 42 ++++++++---------------------------------- 1 file changed, 8 insertions(+), 34 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 3378cf36166dae..fc24802a76e4bc 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -443,41 +443,15 @@ referenced by the ES module entry point are evaluated as ES modules. When an application is using a package that provides both CommonJS and ES module sources, there is a risk of certain bugs if both versions of the package get -loaded (for example, because one version is imported by the application and the -other version is required by one of the application’s dependencies). Such a -package might look like this: - - -```js -// ./node_modules/pkg/package.json -{ - "type": "module", - "main": "./pkg.cjs", - "exports": { - ".": "./pkg.cjs", - "./module": "./pkg.mjs" - } -} -``` - -In this example, `require('pkg')` always resolves to `pkg.cjs`, including in -versions of Node.js where ES modules are unsupported. In Node.js where ES -modules are supported, `import 'pkg/module'` references `pkg.mjs`. - -The potential for bugs comes from the fact that the `pkg` created by `const pkg +loaded. This potential comes from the fact that the `pkg` created by `const pkg = require('pkg')` is not the same as the `pkg` created by `import pkg from -'pkg/module'`. This is the “dual package hazard,” where two versions of the same -package can be loaded within the same runtime environment. While it is unlikely -that an application or package would intentionally load both versions directly, -it is common for an application to load one version while a dependency of the -application loads the other version. This hazard can happen because Node.js -supports intermixing CommonJS and ES modules, and can lead to unexpected -behavior. - -The hazard is also present when [Conditional Exports][] with the -`--experimental-conditional-exports` flag are used. In that case, instead of -`'pkg'` and `'pkg/module'` as in the example above, both `require` and `import` -would use `'pkg'`. The hazard is the same. +'pkg'` (or an alternative main path like `'pkg/module'`). This is the “dual +package hazard,” where two versions of the same package can be loaded within the +same runtime environment. While it is unlikely that an application or package +would intentionally load both versions directly, it is common for an application +to load one version while a dependency of the application loads the other +version. This hazard can happen because Node.js supports intermixing CommonJS +and ES modules, and can lead to unexpected behavior. If the package main export is a constructor, an `instanceof` comparison of instances created by the two versions returns `false`, and if the export is an From 1f7b577c01c808a31e11d40fe30eac0083eeb30b Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Mon, 11 Nov 2019 21:12:17 -0800 Subject: [PATCH 3/9] doc: esm: revisions per feedback --- doc/api/esm.md | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index fc24802a76e4bc..bec7e2cb270592 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -499,6 +499,11 @@ create an ES module wrapper file that defines the named exports. Using ES module wrapper is used for `import` and the CommonJS entry point for `require`. +> Note: While `--experimental-conditional-exports` is flagged, a package +> using this pattern will throw when loaded via `require()` in modern +> Node.js, unless package consumers use the `--experimental-conditional-exports` +> flag. + ```js // ./node_modules/pkg/package.json @@ -553,13 +558,13 @@ This approach is appropriate for any of the following use cases: * The package stores internal state, and the package author would prefer not to refactor the package to isolate its state management. See the next section. -A variant of this approach would add an export, e.g. `"./module"`, to point to -an all-ES module-syntax version the package. This could be used via `import +A variant of this approach not requiring `--experimental-conditional-exports` +for consumers could be to add an export, e.g. `"./module"`, to point to an +all-ES module-syntax version the package. This could be used via `import 'pkg/module'` by users who are certain that the CommonJS version will not be loaded anywhere in the application, such as by dependencies; or if the CommonJS version can be loaded but doesn’t affect the ES module version (for example, -because the package is stateless). This variant would not require -`--experimental-conditional-exports`: +because the package is stateless): ```js @@ -574,6 +579,11 @@ because the package is stateless). This variant would not require } ``` +If the `--experimental-conditional-exports` flag is dropped and therefore +[Conditional Exports][] become available without a flag, this variant could be +easily updated to use conditional exports by adding conditions to the `"."` +path; while keeping `"./module"` for backward compatibility. + ##### Approach #2: Isolate State The most straightforward `package.json` would be one that defines the separate @@ -659,6 +669,28 @@ This approach is appropriate for any of the following use cases: Even with isolated state, there is still the cost of possible extra code execution between the CommonJS and ES module versions of a package. +As with the previous approach, a variant of this approach not requiring +`--experimental-conditional-exports` for consumers could be to add an export, +e.g. `"./module"`, to point to an all-ES module-syntax version the package: + + +```js +// ./node_modules/pkg/package.json +{ + "type": "module", + "main": "./index.cjs", + "exports": { + ".": "./index.cjs", + "./module": "./index.mjs" + } +} +``` + +If the `--experimental-conditional-exports` flag is dropped and therefore +[Conditional Exports][] become available without a flag, this variant could be +easily updated to use conditional exports by adding conditions to the `"."` +path; while keeping `"./module"` for backward compatibility. + ## import Specifiers ### Terminology From c9b6df34e59ef3ce402c83a6c6167d5d3d5982d1 Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Tue, 12 Nov 2019 22:03:51 -0800 Subject: [PATCH 4/9] doc: esm: give "main" a heading, mention other entry points --- doc/api/esm.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index bec7e2cb270592..5fc1a219f34e1a 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -184,6 +184,8 @@ unspecified. ### Package Entry Points +#### package.json "main" + The `package.json` `"main"` field defines the entry point for a package, whether the package is included into CommonJS via `require` or into an ES module via `import`. @@ -219,7 +221,12 @@ The `"main"` field can point to exactly one file, regardless of whether the package is referenced via `require` (in a CommonJS context) or `import` (in an ES module context). -### Package Exports +[Package Exports][] provide an alternative to `"main"` where the package main +entry point can be defined while also encapsulating the package, preventing any +other entry points besides those defined in `"exports"`. If package entry points +are defined in both `"main"` and `"exports"`, the latter takes precedence. + +#### Package Exports By default, all subpaths from a package can be imported (`import 'pkg/x.js'`). Custom subpath aliasing and encapsulation can be provided through the @@ -1385,6 +1392,7 @@ success! [ECMAScript-modules implementation]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md [ES Module Integration Proposal for Web Assembly]: https://github.com/webassembly/esm-integration [Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md +[Package Exports]: #esm_package_exports [Terminology]: #esm_terminology [WHATWG JSON modules specification]: https://html.spec.whatwg.org/#creating-a-json-module-script [`data:` URLs]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs From 2637f4dbb23bac4df5fbaea52f0a5f5bc083e5a6 Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Wed, 13 Nov 2019 20:38:02 -0800 Subject: [PATCH 5/9] doc: esm: add intro to package entry points section --- doc/api/esm.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 5fc1a219f34e1a..f3cdd124985762 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -184,6 +184,18 @@ unspecified. ### Package Entry Points +There are two fields that can define entry points for a package: `"main"` and +`"exports"`. The `"main"` field is supported in all versions of Node.js, but its +capabilities are limited: it only defines the main entry point of the package. +The `"exports"` field can also be used to define the main entry point of the +package, as well as other defined entry points; and the package can be +encapsulated, so that extra effort is required to reference files within the +package that aren’t the defined public API. `"exports"` can also map an entry +point to different files per environment, for example for all environments +versus browser environments; and with `--experimental-conditional-exports` +`"exports"` can define separate files for Node.js CommonJS and ES module +environments. + #### package.json "main" The `package.json` `"main"` field defines the entry point for a package, @@ -224,7 +236,10 @@ ES module context). [Package Exports][] provide an alternative to `"main"` where the package main entry point can be defined while also encapsulating the package, preventing any other entry points besides those defined in `"exports"`. If package entry points -are defined in both `"main"` and `"exports"`, the latter takes precedence. +are defined in both `"main"` and `"exports"`, the latter takes precedence in +versions of Node.js that support `"exports"`. [Conditional Exports][] can also +be used within `"exports"` to define different package entry points per +environment. #### Package Exports From 4b75b64cae76489eb1dff70f6acfc4f4f5db1bb7 Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Wed, 13 Nov 2019 20:40:16 -0800 Subject: [PATCH 6/9] doc: rename for clarity --- doc/api/esm.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index f3cdd124985762..6812447fe17f30 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -455,7 +455,7 @@ ignores) the top-level `"module"` field. Node.js can now run ES module entry points, and a package can contain both CommonJS and ES module entry points (either via separate specifiers such as -`'pkg'` and `'pkg/module'`, or both at the same specifier via [Conditional +`'pkg'` and `'pkg/es-module'`, or both at the same specifier via [Conditional Exports][] with the `--experimental-conditional-exports` flag). Unlike in the scenario where `"module"` is only used by bundlers, or ES module files are transpiled into CommonJS on the fly before evaluation by Node.js, the files @@ -465,20 +465,21 @@ referenced by the ES module entry point are evaluated as ES modules. When an application is using a package that provides both CommonJS and ES module sources, there is a risk of certain bugs if both versions of the package get -loaded. This potential comes from the fact that the `pkg` created by `const pkg -= require('pkg')` is not the same as the `pkg` created by `import pkg from -'pkg'` (or an alternative main path like `'pkg/module'`). This is the “dual -package hazard,” where two versions of the same package can be loaded within the -same runtime environment. While it is unlikely that an application or package -would intentionally load both versions directly, it is common for an application -to load one version while a dependency of the application loads the other -version. This hazard can happen because Node.js supports intermixing CommonJS -and ES modules, and can lead to unexpected behavior. +loaded. This potential comes from the fact that the `pkgInstance` created by +`const pkgInstance = require('pkg')` is not the same as the `pkgInstance` +created by `import pkgInstance from 'pkg'` (or an alternative main path like +`'pkg/module'`). This is the “dual package hazard,” where two versions of the +same package can be loaded within the same runtime environment. While it is +unlikely that an application or package would intentionally load both versions +directly, it is common for an application to load one version while a dependency +of the application loads the other version. This hazard can happen because +Node.js supports intermixing CommonJS and ES modules, and can lead to unexpected +behavior. If the package main export is a constructor, an `instanceof` comparison of instances created by the two versions returns `false`, and if the export is an -object, properties added to one (like `pkg.foo = 3`) are not present on the -other. This differs from how `import` and `require` statements work in +object, properties added to one (like `pkgInstance.foo = 3`) are not present on +the other. This differs from how `import` and `require` statements work in all-CommonJS or all-ES module environments, respectively, and therefore is surprising to users. It also differs from the behavior users are familiar with when using transpilation via tools like [Babel][] or [`esm`][]. From a0d7c997637a647231862025c2c70908954a4592 Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Wed, 13 Nov 2019 21:11:27 -0800 Subject: [PATCH 7/9] doc: merge intro language --- doc/api/esm.md | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index 6812447fe17f30..d4a9b22ea775c1 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -187,14 +187,13 @@ unspecified. There are two fields that can define entry points for a package: `"main"` and `"exports"`. The `"main"` field is supported in all versions of Node.js, but its capabilities are limited: it only defines the main entry point of the package. -The `"exports"` field can also be used to define the main entry point of the -package, as well as other defined entry points; and the package can be -encapsulated, so that extra effort is required to reference files within the -package that aren’t the defined public API. `"exports"` can also map an entry -point to different files per environment, for example for all environments -versus browser environments; and with `--experimental-conditional-exports` -`"exports"` can define separate files for Node.js CommonJS and ES module -environments. +The `"exports"` field, part of [Package Exports][], provides an alternative to +`"main"` where the package main entry point can be defined while also +encapsulating the package, preventing any other entry points besides those +defined in `"exports"`. If package entry points are defined in both `"main"` and +`"exports"`, the latter takes precedence in versions of Node.js that support +`"exports"`. [Conditional Exports][] can also be used within `"exports"` to +define different package entry points per environment. #### package.json "main" @@ -233,14 +232,6 @@ The `"main"` field can point to exactly one file, regardless of whether the package is referenced via `require` (in a CommonJS context) or `import` (in an ES module context). -[Package Exports][] provide an alternative to `"main"` where the package main -entry point can be defined while also encapsulating the package, preventing any -other entry points besides those defined in `"exports"`. If package entry points -are defined in both `"main"` and `"exports"`, the latter takes precedence in -versions of Node.js that support `"exports"`. [Conditional Exports][] can also -be used within `"exports"` to define different package entry points per -environment. - #### Package Exports By default, all subpaths from a package can be imported (`import 'pkg/x.js'`). From 94e85ab2244908e9e71cad8c0955c363b7690d5b Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Sat, 16 Nov 2019 11:50:25 -0800 Subject: [PATCH 8/9] doc: typo Co-Authored-By: Vse Mozhet Byt --- doc/api/esm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index d4a9b22ea775c1..ba9577eec9018d 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -685,7 +685,7 @@ execution between the CommonJS and ES module versions of a package. As with the previous approach, a variant of this approach not requiring `--experimental-conditional-exports` for consumers could be to add an export, -e.g. `"./module"`, to point to an all-ES module-syntax version the package: +e.g. `"./module"`, to point to an all-ES module-syntax version of the package: ```js From 912f28183555cd79903eacf2ea57cb14d7139e28 Mon Sep 17 00:00:00 2001 From: Geoffrey Booth Date: Sat, 16 Nov 2019 11:50:41 -0800 Subject: [PATCH 9/9] doc: typo Co-Authored-By: Vse Mozhet Byt --- doc/api/esm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/esm.md b/doc/api/esm.md index ba9577eec9018d..965a995c5e72c4 100644 --- a/doc/api/esm.md +++ b/doc/api/esm.md @@ -574,7 +574,7 @@ This approach is appropriate for any of the following use cases: A variant of this approach not requiring `--experimental-conditional-exports` for consumers could be to add an export, e.g. `"./module"`, to point to an -all-ES module-syntax version the package. This could be used via `import +all-ES module-syntax version of the package. This could be used via `import 'pkg/module'` by users who are certain that the CommonJS version will not be loaded anywhere in the application, such as by dependencies; or if the CommonJS version can be loaded but doesn’t affect the ES module version (for example,