doc: merge CJS and ESM docs

[aduh95]

Based on @GeoffreyBooth's suggestion in #32970 (comment).

Documents package.json supported fields.

Fixes: #33143
Alternate to #32970

Problem this PR is trying to solve

Node.js has recently started to rely more on the package.json file, and I feel it would be helpful to have a separate document defining the fields we use (a bit like the guide on npm docs).

Currently, there are features supported by both ESM and CJS loaders, and it is documented in one arbitrary file (E.G.: Conditional Exports is documented in esm.md only, Source Maps API in modules.md only, …).

This PR merges the two documents (as well as the package.json documentation from #32970) in the modules.md file.

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• documentation is changed or added
• commit message follows commit guidelines

[GeoffreyBooth]

Thanks for doing this. Because the diff is huge, and GitHub doesn't make clear what text was simply moved as opposed to moved and modified, do you mind listing which sections in the new modules.md were unedited moves from esm.md and which had changes? I assume most will be the former.

[devsnek]

Splitting it into separate commits might help. We will also need some sort of redirect from esm.html which preserves the # fragment.

[aduh95]

I've split it into several commits as @devsnek suggested, each one with a small enough diff for GitHub UI.

We will also need some sort of redirect from esm.html which preserves the # fragment.

That would be on Node.js website side, right?

which sections in the new modules.md were unedited moves from esm.md and which had changes?

Some section have changed names:

• modules.md ---> modules.md
  • Addenda: Package Manager Tips --> Tips for Package Manager Authors
  • The Module object --> Utility Methods
  • All Together... --> CommonJS Modules Resolution Algorithm
• esm.md ---> modules.md
  • Enabling ---> Usage
  • Resolution Algorithm ---> ECMAScript Modules Resolution Algorithm

The content of the sections has not changed (except minor stuff like links and eslint comments).

[guybedford]

For this to land I think we need to ensure the following:

• It should start with the ECMAScript modules information with CommonJS at the end, as most beginner users will likely want to start with that and we shouldn't make them scroll all the way down to get the info they need on basic authoring in Node.js.
• We should aim to ensure that the first section has a copy and past example that is enough to get a beginner user going with ECMAScript modules.

I still have some concerns about putting everything together in one big doc page from a maintenance and digestibility point of view, but I can certainly get behind it if others agree it is cleaner.

[aduh95]

Hey, going back to this PR, it seems it requires a non negligible amount of work to fix the merge conflicts and have it ready for review. I'd like to know if the general feeling among Node.js collaborators is that this idea is worth pursuing before investing my time in it. Specifically, some concerns have been raised regarding the length of the resulting document, is this going to be a deal breaker?

Based on the comments from @GeoffreyBooth and @guybedford, here is the structure I'm aiming for:

TOC

Modules

• Introduction (new, explain that Node.js has two module systems)
  • ECMAScript modules (based on esm.md introduction)
  • CommonJS modules (based on modules.md introduction)
• Using (based on esm.md section “Enabling”)
  • package.json "type" field
  • Package Scope and File Extensions
  • --input-type flag
• ECMAScript modules (new; all subsections from esm.md)
  • import Specifiers
    • Terminology
      • data: imports
  • import.meta
  • Differences between ES modules and CommonJS
    • Mandatory file extensions
    • No NODE_PATH
    • No require, exports, module.exports, __filename, __dirname
    • No require.resolve
    • No require.extensions
    • No require.cache
    • URL-based paths
  • Interoperability with CommonJS
    • require
    • import statements
    • import() expressions
  • CommonJS, JSON, and Native modules
  • Builtin modules
  • Experimental JSON modules
  • Experimental Wasm modules
  • Experimental top-level await
  • Experimental loaders
    • Hooks
      • resolve hook
      • getFormat hook
      • getSource hook
      • transformSource hook
      • getGlobalPreloadCode hook
      • dynamicInstantiate hook
    • Examples
      • HTTPS loader
      • Transpiler loader
• CommonJS modules (new; all subsections from modules.md)
  • The module wrapper
  • The module scope
    • __dirname
    • __filename
    • exports
    • module
    • require(id)
      • require.cache
      • require.extensions
      • require.main
      • require.resolve(request[, options])
        • require.resolve.paths(request)
  • The module object
    • module.children
    • module.exports
      • exports shortcut
    • module.filename
    • module.id
    • module.loaded
    • module.parent
    • module.paths
    • module.require(id)
  • Accessing the main module
  • Core modules
  • File modules
    • require of files with the .mjs extension (from modules.md “Addenda: The .mjs extension”)
  • Folders as modules
  • Loading from the global folders
  • Loading from node_modules folders
  • Cycles
  • Caching
    • Module caching caveats
• Packages (from esm.md)
  • The package.json file (new section introducing the file itself, with very brief subsections that are nothing more than links to the appropriate sections)
    • "name"
    • "exports"
    • "main"
    • "type"
  • Package entry points
    • Main entry point export
    • Subpath exports
    • Package exports fallbacks
    • Exports sugar
    • Conditional exports
    • Nested conditions
    • Self-referencing a package using its name
  • Dual CommonJS/ES Module Packages
    • Dual package hazard
    • Writing dual packages while avoiding or minimizing hazards
      • Approach deps: update openssl to 1.0.1j #1: Use an ES module wrapper
      • Approach test: don't remove empty.txt on win32 #2: Isolate state
  • Tips for package manager authors (from modules.md “Package Manager Tips”)
• Utility methods (from modules.md “The Module Object”)
  • module.builtinModules
  • module.createRequire(filename)
  • module.createRequireFromPath(filename)
  • module.syncBuiltinESMExports())
• Source map v3 support (from modules.md)
  • module.findSourceMap(path[, error])
  • Class: module.SourceMap
    • new SourceMap(payload)
    • sourceMap.payload
    • sourceMap.findEntry(lineNumber, columnNumber)
• Resolution algorithms (new)
  • ECMAScript modules resolution algorithm (from esm.md section “Resolution Algorithm”)
    • Features
    • Resolver algorithm
    • Customizing ESM specifier resolution algorithm
  • CommonJS resolution algorithm (from modules.md section “All Together...”)

[guybedford]

The more I think about this the more I think it's the wrong call to combine the CJS and ESM documentation. I think in general we should be splitting API pages up into smaller chunks not combining them into larger ones.

It seems unnecessary for users writing ES modules to need to use the same documentation page as the CommonJS docs which are also much more well established and have URLs and links that shouldn't be changed.

Perhaps we can take this same initiative in the other direction - and split out the shared content into its own page rather? I liked the proposal previously of a separate package.json API page that is used as the point of linkage between CJS and ESM resolution.

@aduh95 do you think your outline could adapt to that sort of approach here, or does that seem too big of a shift? Sorry for not being clearer with my opinion from the start here. Hard questions! Feedback from others welcome too!

[aduh95]

@guybedford I tend to agree with you, especially if we take into account the maintainability of it all.

What do you all think of creating a new CommonJS modules page and move all CJS specific docs there? I would also love to have feedback from more people!

[GeoffreyBooth]

What do you all think of creating a new CommonJS modules page and move all CJS specific docs there? I would also love to have feedback from more people!

We could maybe rename the pages like “Modules: CommonJS” and “Modules: ECMAScript” so that they’re next to each other in the table of contents.

I discussed with @guybedford a bit and we were thinking that maybe the page for common stuff could be organized around information for package authors? So like “Packages” or “Modules: Authoring Packages” or somesuch. Then it doesn’t need to be so closely tied to package.json; the information about "type" and "main" can stay on the respective CommonJS and ESM pages, as those fields deserve to be documented on each page (and behave differently in each system). "name" and "exports" and "imports" can be documented on this new Packages page, as well as everything else in your outline above under your Packages heading (entry points, hazard).

This would leave source maps as the odd section out, that's used in both module systems but unrelated to authoring packages. We're thinking that the source maps info doesn't really belong on any of these pages, as it's not really related to modules or packages at all; it could maybe move to the Debugger page or even its own top-level page.

[guybedford]

@aduh95 thanks, I'm glad to hear you don't find the feedback too disheartening!

We could maybe rename the pages like “Modules: CommonJS” and “Modules: ECMAScript” so that they’re next to each other in the table of contents.

This makes sense. Ideally we would set up a redirect for modules.html to redirect to modules-commonjs.html or whatever the link would be. It would be important to ensure this as part of the process.

Perhaps lets open a new issue with the proposal for a Modules: CommonJS, Modules: ECMAScript and Modules: Package Configuration or whatever we want to call them exactly with a rough outline and try to get wider feedback there? We can also add this as an agenda item for the modules group if we want to get wider docs feedback discussion as well.

[guybedford]

Also what would happen to the docs on the require('module') builtin? Perhaps that should remain its own page?

[aduh95]

Closing this, see nodejs/modules#539 for more context.