Determining Module Migration and Shipping Strategy

[DanielRosenwasser]

The work to get the TypeScript codebase over to modules still has a few open questions. There are a few high-level things we want to think about. We want to be able to maximize a few things:

• development experience on our team
  • We generally feel the setup we have today is okay. Can we get some more build incrementality from modules? Can we avoid regressing in some other way? Are there wins to be had?
• dog-fooding
  • Our team does not always share the same code authoring experience as most users, since we don't use modules. Can we change this?
• direct developer experience
  • TypeScript developers who are setting up their own builds and running TS either directly or through a tool like ts-loader should still have a good time getting things working, and ideally it shouldn't change.
• library consumer experience
  • We shouldn't regress the experience for people importing from the typescript package. Existing import targets shouldn't change, or at least, they shouldn't
• minimizal dependency size
  • Part of the goal is to reduce redundancy between tsc.js, typescript.js, tsserver.js, typescriptServices.js, tsserverlibrary.js, typingsInstaller.js, etc.

What questions/problems does that leave us with?

Internal Naming Hazards

TypeScript defines its own entities that clash with those often defined globally (e.g. Node, Symbol, Map, and perhaps a few more). How can we avoid using the wrong ones?

• The global Node type tends not to be available, as we don't compile against lib.dom.d.ts
• The global Symbol type tends to be incompatible with any places we need a TypeScript symbol.
• Map and Set are actually structurally compatible, and has been a hazard.

Perhaps we'll have to create a lint rule here; however, at some point it would be nice to just drop our internal shims like Map and Set. Maybe for 5.0?

Internal Organization

While TypeScript under namespaces is organized across files in a (usually) logical way, everything is still defined in the same effective scope. That means that (usually) no file ever has to think about what file another function or type comes from. We could preserve this by re-exporting everything from a top-level "barrel", but it's not clear how tenable this is. @jakebailey has more detail, but has been considering backing away from this strategy.

Module Format

Today, TypeScript ships CommonJS modules that also plop a global into scope named ts.

Do we want to continue supporting CommonJS targets? Do we want to ship both CommonJS and ECMAScript modules? Do we want to ship both?

I think it feels obvious to say that we need to at least continue shipping CommonJS. TypeScript is so widely used, and we would be leaving a lot of users stranded if we did this now. Down the road, we could switch to ESM only, but for now it's not hard for us to ship both since we don't have dependencies. While feasible, it obviously diminishes the wins for reducing TypeScript's package size on npm, and we have to worry about divergences. If we ship both CJS and ESM, will we feel confident that they will behave identically?

Bundling

We would prefer to ship fewer files with TypeScript to avoid resolution time if possible. We also don't want to worry about "deep import" problems if we end up supporting older versions of Node.js. All signs here lead to bundling. TypeScript currently doesn't provide the capabilities to bundle a project. Instead, projects like Webpack, Rollup, esbuild, swc, Rome, Parcel, and more have filled in the gap here.

Bundling for Testing

Do we want to bundle ourselves when running tests? If so, do we want our bundler to perform the downlevel compilation on TypeScript itself, or do we want the bundler to run on TypeScript's self-produced output? Do we want to use the same bundler between development and production?

At each step, we risk some amount of divergence. Bundling can possibly lead to different semantics compared to running modules directly. Running two different bundlers between dev and prod can lead to diferent semantics. Using a bundler that downlevel-compiles can lead to different semantics from running TypeScript's output code.

We could make our development and production builds identical (they are today anyway, and this makes things nicely predictable). To do this, we would just have TypeScript produce emit, and have a bundler run on the output (which means it's just one extra step). This has the nice benefit of not worrying about CI contexts vs. local contexts, etc. You can produce the same build of TypeScript no matter where you are, no matter what your system environment variables are.

What's the opportunity cost? Well tools like esbuild are fast! You could imagine starting to run tests before the LKG (last-known-good version) of TypeScript has even finished type-checking itself.

Declaration Bundling

It's not enough to say that we want to ship bundled JavaScript files; any .js files that we ship need to have a corresponding .d.ts files. Currently the solution in this space is to use API Extractor which provides functionality for bundling TypeScript declaration files (a.k.a. ".d.ts rollup"). We would likely run API Extractor over our bundles and also use it as part of our baselined .d.ts test (generated here).

[Jack-Works]

Do we want to continue supporting CommonJS targets? Do we want to ship both CommonJS and ECMAScript modules? Do we want to ship both?

I'm curious, how is it possible to emit both CJS and ESM for a single codebase, with only TSC. They're both emitted as .js files and cannot be interpreted as both ESM and Commonjs within a package.

[jakebailey]

You emit twice, via two tsconfigs (or two invocations with differing flags). We already have different variants of tsconfigs to do things like distinguish a release build from a dev build.

[jakebailey]

For those following this issue, the module conversion PR has been posted: #51387