Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A Proposal For Module Resolution #50152

Closed
andrewbranch opened this issue Aug 2, 2022 · 72 comments
Closed

A Proposal For Module Resolution #50152

andrewbranch opened this issue Aug 2, 2022 · 72 comments
Assignees
Labels
In Discussion Not yet reached consensus Suggestion An idea for TypeScript

Comments

@andrewbranch
Copy link
Member

Background

When a user writes a module specifier (the string literal after from in an import declaration) in a TypeScript file, how should the compiler resolve that string to a file on disk to be included in type checking? Because TypeScript never rewrites module specifiers in its JavaScript emit, the only possible answer is that it should mirror whatever resolution behavior the code’s intended runtime module resolver has. I’m using “runtime module resolver” to mean the system whose module resolution behavior has observable effects at runtime: it may be a component of the runtime itself, as in Node, or it may be a bundler that consumes the module specifiers to produce one or more script files. (The “runtime” distinction is made to exclude analysis tools like linters, which may perform module resolution without having any impact on runtime behavior.) TypeScript’s way of handling this has been to say that the user must indicate what their code’s runtime module resolver is via the moduleResolution compiler option so the compiler can mirror it.

As little as five years ago, there were only two places JavaScript could run that were worth mentioning: in Node as CommonJS modules, and in the browser as scripts. For the former, TypeScript had --moduleResolution node. (The latter needed no moduleResolution mode, though you can argue some sort of none value would have been appropriate.) However, bundlers like Webpack were widely used and were themselves module resolvers, perhaps deserving their own moduleResolution setting according to TypeScript’s philosophy. But demand for bundler-specific module resolution was essentially nonexistent, because bundlers mostly just copied Node’s module resolution algorithm, so users were able to get by with --moduleResolution node.

(Note: as of this writing, --moduleResolution node16 and --moduleResolution nodenext are identical in TypeScript. The latter is intended to be updated as Node changes. For brevity, I use node16 in this writing, but both are equally applicable everywhere.)

Over the next few years, though, the landscape changed. Browsers adopted ESM as a natively supported format, and Node added ESM support alongside CJS, with a complex interop system and new features like package.json exports. A new wave of bundlers and runtimes emerged, and many adopted some of the features that Node introduced. But this time, none was similar enough to Node to piggyback on TypeScript’s --moduleResolution node16 option without users noticing problems.

Today

In this new landscape, users have been trying both node and node16 with bundlers and browsers and hitting walls, which I will explore in some detail. In brief, the JavaScript ecosystem is in a phase where we cannot hope to provide a dedicated moduleResolution mode for every runtime and bundler. At the same time, we have resisted allowing resolver plugins for many reasons:

  • security and performance concerns
  • implementing correct module specifier generation for a given resolution mode for declaration emit, auto-imports, and path completions is extremely non-trivial
  • concerns about reliability under file-watching modes (a resolution must return not just what it found, but where it looked and found nothing, and where it looked up auxiliary information used in resolution such as package.json files)
  • we would like to avoid allowing bespoke per-project resolution simply to reduce chaos

This philosophy has brought TypeScript to a point where we have avoided some significant pitfalls, but have essentially no support for module resolvers that are not Node. To make the issues explicit, let’s examine some hypothetical case studies.

Bundling with Webpack, esbuild, or Vite

These bundlers use a Node-CJS-like resolution algorithm and support package.json exports. If the user chooses --moduleResolution node, any of their dependencies that modify their export structure via package.json exports will be misrepresented by TypeScript—imports from that package may not resolve, they may resolve to incorrect files, and they will receive incorrect auto-imports and path completions. If the user chooses --moduleResolution node16, TypeScript will resolve their imports against dependencies’ package.json exports, but the conditions it uses in the lookup may be wrong: bundlers always set the import condition for imports and the require definition for require calls, but TypeScript believes that import declarations in files that have not been explicitly scoped as ESM will be transpiled into require calls, so it looks up these imports with the require condition, which could lead to incorrect resolutions. Moreover, in these files, TypeScript will prohibit imports (because it think they are actually requires) of ESM-format files. The bundler has no such restriction, as its CJS/ESM divide is purely syntactic. (This is a slight oversimplification and the three bundlers mentioned behave slightly differently, but the simplification is good enough for describing the user experience.) If the user tries to get around this by scoping all of their files as ESM by setting "type": "module" in their own package.json, TypeScript will impose Node’s much stricter ESM resolution algorithm on those files, disabling index-file resolution and extensionless lookups—in fact, the extension the user has to write is .js, which will be nonsensical for the context, where the runtime module resolver (the bundler) only ever sees .ts files. (Vite and esbuild tolerate this extension mismatch out of the box; Webpack has historically required a plugin but just added a config setting for it.) This configuration satisfies both the bundler and TypeScript, but at a high DX cost for the user—TypeScript imposes rules on resolution that are wholly unnecessary for the bundler.

Running in Bun

The situation is exactly the same as the above, since Bun’s module resolver is a port of esbuild’s, and it consumes TS files directly.

Bundling with Parcel or Browserify

These bundlers do not (yet) support package.json exports, so --moduleResolution node is still a reasonably good fit.

Writing ESM for the browser

Every module resolution mode except classic performs node_modules resolution, which does not happen in the browser. classic performs index-file and extensionless lookups, which does not happen in the browser. The closest the user can get is probably to use node16 such that index-file and extensionless lookups are disabled, but they have to take care to avoid node_modules lookups and importing CommonJS dependencies.

Writing ESM for Node, browser, or Deno

We have heard a few arguments recently about the ability to write code targeting multiple runtimes, mostly in the form of “if you let me write my imports with .ts extensions and emit them as .js extensions, my input files will work in Deno and my output files will work in Node” which is not generally true. However, it is true that Node, the browser, and Deno have a small amount of overlap in resolution behavior such that it is possible to write ES modules in JS and publish them both to npm and to a CDN where they can be consumed by browsers or Deno. A user trying to do this today faces the same situation as the case above, since the overlap between these systems is just relative URL imports including extensions: there is no mode restrictive enough to avoid writing imports that will work in Node but not in Deno or the browser. (Note that targeting a single bundler which produces a separate output for each target runtime is, for now, a better approach for multi-platform JS authoring.)

Proposal

Existing module resolution modes (with the exception of classic, whose existence is still a mystery to me) have intended to target one specific runtime and have been named for that runtime—node (v11 and before), node16, and nodenext—and the resolution features they entail are non-configurable implementation details. To move forward, I suggest a strategy of composition: expose some lower-level modes that can be combined with additional options to build up modes that are suitable for a variety of runtime resolvers. If the ecosystem converges on combinations of these settings, we can encapsulate them in a named mode. To start this process, I propose the following reorganization and expansion of options (all names subject to bikeshedding):

Module Resolution Modes

  1. Expose what is now called node as a low-level mode called conventional, with a slight modification necessary to support .ts extension resolution under noEmit, and defaulting esModuleInterop to true. (The name, which I am more than happy to change, is a reference to the fact that most runtimes and bundlers have copied node_modules resolution, extensionless lookups, and special index-file handling from Node to the point where users no longer think of these features as specific to Node. Since we now have node16 which is highly Node-specific, the goal is to create a situation where the only people who should choose a moduleResolution option named after Node are people who are actually using Node.)
  2. Implement today’s node as a composition of conventional and an internal-only option that undoes the modifications mentioned in (1) to preserve backward compatibility. Also, deprecate the name node in favor of node-legacy to encourage users of modern Node to consider migrating to node16, and to encourage users of other runtimes and bundlers to consider migrating to conventional. Today’s node is only accurate to Node v11 and earlier, so we need to start guiding people away from it at some point.
  3. Implement a new low-level mode called minimal which resolves only relative module specifiers including file extensions, and attempts to parse all files as ESM. This can be used as a base for browser module resolution.
  4. Leave classic, node16, and nodenext as they are.

Module Resolution Options

  1. Add an option to enable/disable package.json exports in conventional and add conditions to the resolver. (May also apply to other modes that do node_modules resolution, i.e. everything but classic and minimal.)

That’s it for now—in the future, import maps, HTTP imports, and other features adopted by more than one runtime resolver should be exposed as options. When/if we have support for import maps and HTTP imports specifically, we should consider creating a mode named browser that is a composition of minimal and those options defaulted to true.

Resolution of relative module specifiers ending in .ts

Both minimal and conventional (but not node-legacy) will support resolution to .ts files by specifying a .ts extension in the module specifier. This will be an error, as it is today, unless noEmit is enabled. This allows users who are bundling or directly running their TypeScript source to write relative module specifiers with the extension that their runtime module resolver will actually see, which has always been the underlying goal of telling users to write .js extensions when their runtime resolver will operate on the emitted JS code. Additionally, in these modes, I suggest that it be legal to write an import type of a module specifier ending in .d.ts.

This cannot be supported in today’s node or node16 in a fully backward-compatible way. In these modes, an import of "./foo.ts" will resolve to foo.ts.js or foo.ts.d.ts in the same directory even if foo.ts is also present; unsupported extensions are not probed for existence before moving on to fallbacks. This amounts to a bug in node and node16. I have proposed to leave the bug in place for node-legacy to preserve backward compatibility, but it may be reasonable to try fixing it everywhere and listen for feedback.

It should be noted that composite projects may not disable emit if they are referenced by another project. It may be possible to relax the noEmit restriction to emitDeclarationOnly. The primary challenge here is a portability concern: older moduleResolution modes will not be able to resolve the .ts-suffixed specifiers in those declaration files. I think it’s worth fixing node16 to support this; they would receive the bug fix described above so that they can always resolve .ts-suffixed specifiers, but would continue to issue a checker error. That way, we can safely silence the error in declaration files for better portability, and projects that need to use .ts-suffixed imports could be composite project references. Fixing node16 in this way would also prepare us for the possibility of Node running directly on TS files, transpiling in-memory like ts-node, an idea that has been gaining traction with Node maintainers recently.

However, I think much of the demand we’ve heard so far for being able to use .ts-suffixed module specifiers has been misplaced. Users who tried to use node16 with a bundler may have been prompted to add a .js extension to a module specifier and thought that adding a .ts extension makes more sense, when in actuality they can continue to use extensionless imports in a mode like conventional. Others demand .ts-suffixed imports in combination with module specifier rewriting because they believe that will let them write input code that will run natively in Deno, while tsc’s output will run natively in Node. This is out of scope; the way to write once and ship to multiple environments is to target a bundler and produce multiple bundles. Consequently, I think there are very few users who need to write .ts-suffixed imports (especially in a world with conventional), but they are unobjectionable in noEmit and easy to implement. The feature is not core to this proposal, but I believe it would be a mistake to create new module resolution modes without at least fixing the aforementioned bug to carve out the possibility of .ts-suffixed imports resolving in the future.

Notes on conventional

This proposal does not allow for a perfect mapping of TypeScript’s resolution behavior onto every bundler, but I think it covers most cases, or what I will call all reasonable cases. If we wanted to be a bit prescriptive, I would be tempted to prohibit .cts and .cjs files, disallow import m = require(...) syntax in TypeScript files, and disable resolution of require calls in JS files. Some of the newer bundlers are explicitly ESM-only, ignoring or prohibiting require calls in user code and converting library dependencies from CJS to ESM. No bundler I tested had separate CJS and ESM resolution algorithms, with the exception of setting import vs. require in the resolver conditions when looking up package.json exports. There seems to be little reason to allow explicitly CJS constructs in implementation files in this mode (while CJS constructs in dependency declaration files obviously need to be consumable). As in TS files today, users will still be free to write require calls, but they will not have special resolution behavior.

Unanswered questions

  • This proposal exposes users to increased complexity. I don’t see a way around that. I have also proposed deprecating --moduleResolution node, which is the default for --module commonjs. Consequently, many users are using node without realizing it. This raises the question of what defaults we should have in the future, what module settings should be allowed with these moduleResolution modes, and more broadly, how to guide users into selecting the correct settings for their project.
    • A possible mitigation is to allow multiple tsconfig.json extends and encourage bundlers to publish (or publish ourselves under @typescript) tsconfig bases that reflect the resolution behaviors supported by these bundlers out-of-the-box. That way, an esbuild user could write
      {
            "extends": ["webpack", "./tsconfig.base.json"],
            "compilerOptions": { /* ... */ }
          }
  • Should .ts-suffixed resolution be automatically allowed based on noEmit, or should it be gated behind another flag, or should the capability be preserved for the future but not enabled yet? It makes sense to me that noEmit should enable it, because if you’re writing modules but not emitting, it stands to reason that another tool is going to consume the TS modules that you wrote. @DanielRosenwasser raised the idea that this may cannibalize project references usage, which requires declarations to be emitted and can help speed up type checking when splitting large codebases. More thought needs to be put into how .ts imports would work with declaration emit.
  • For simplicity and consistency, can we fix the resolution bug where files with unsupported extensions do not stop other lower priority extensions from being looked up even in today’s node, breaking backward compatibility? The first time I considered this, I thought this would be an untenable breaking change, because people rely on this behavior to write declarations for files with unsupported extensions, e.g. import styles from "./styles.css" would resolve to styles.css.d.ts because it thinks that’s analogous to styles.css.js, not styles.css. However, @weswigham has proposed a general solution for this at Proposal: Enable declaration files for non-js-extensioned files #50133. Taking some form of that proposal may be key to fixing this “bug” in any resolution mode without effectively losing a feature.

Related: #37582, #49083, #46452, #46334, and probably a dozen others

@andrewbranch andrewbranch added Discussion Issues which may not have code impact Suggestion An idea for TypeScript In Discussion Not yet reached consensus and removed Discussion Issues which may not have code impact labels Aug 2, 2022
@weswigham
Copy link
Member

Add an option to enable/disable package.json exports 

I think a very important question for this is: With what default conditions (configurable?) and what behavior for the import/require conditions? Each bundler runs different condition sets from what I know - most have a bundler-specific one, in addition to often using a module condition, on top of import and require.

@weswigham
Copy link
Member

weswigham commented Aug 2, 2022

Also, an important concern for declaration emit is cross-resolution mode compatibility. node and node16 are often close enough that we can gloss over the differences in DT, but with more available configuration, will we potentially need multiple package implementations on DT for different module resolution settings, and, if so, how do we ship (or compose) that? (Or are export map conditions alone sufficient?)

@weswigham
Copy link
Member

weswigham commented Aug 3, 2022

Also ref #29353 where I did what I thought would be some minimal emit changes to create a mode about as flexible as bundlers were at the time. That didn't include any resolution changes :)

@andrewbranch
Copy link
Member Author

With what default conditions (configurable?) and what behavior for the import/require conditions?

import or require based on importing syntax (if we allow require to resolve at all); additional conditions should be configurable.

will we potentially need multiple package implementations on DT for different module resolution settings

This is a problem for us and DT just about as much as it’s a problem for actual runtimes and bundlers consuming implementation packages from npm. What’s in DT should reflect what’s in the implementation package on npm. If the implementation package writes imports that only resolve under bizarre resolvers of the future, it will make sense for the DT package to do the same; it will make sense for users to have broken types under node16 if Node 16 will choke on the implementation package. (This is already basically the world we live in but substitute assumptions about resolution with assumptions about globals.) As of today, nothing I’ve suggested really erodes at the rough cross-compatibility between these existing modes (the changes I proposed to make conventional out of node are very minor). In fact, minimal is the ultimate common-denominator mode, so packages aiming to be as cross-compatible as possible can opt into that (if they take on the extreme limitation of vendoring their dependencies and referencing them with relative paths).

@jakebailey
Copy link
Member

With what default conditions (configurable?) and what behavior for the import/require conditions? Each bundler runs different condition sets from what I know - most have a bundler-specific one, in addition to often using a module condition, on top of import and require.

Just to add onto this (I'm sure this is known from the research put into this proposal, so forgive me if it's already been considered), esbuild has conditions totally configurable (https://esbuild.github.io/api/#conditions, https://esbuild.github.io/api/#main-fields); its behavior was almost changed in the last release which would have changed its defaults too in some scenarios to avoid dual-package hazards (evanw/esbuild#2417), since bundlers can sort of just "import" anything however they want regardless of the syntax (since they control the runtime import behavior internally).

@DanielRosenwasser
Copy link
Member

DanielRosenwasser commented Aug 3, 2022

Seems like loaders for Node.js such as ts-node and tsx, or even a future version of Node.js, would possibly want to allow .ts extensions.

Would this be as simple as taking conventional, add the features from node16+, and set noEmit?

I do have some concerns with the approach of enabling .ts extensions without its own flag. Maybe over time noEmit could imply that, the same way that checkJs implied allowJs. But maybe my concerns are not well-founded.

@jakebailey
Copy link
Member

jakebailey commented Aug 3, 2022

Both minimal and conventional (but not node-legacy) will support resolution to .ts files by specifying a .ts extension in the module specifier. This will be an error, as it is today, unless noEmit is enabled.

I'm curious how this will play out with projects that interpret TS directly; I seem to recall one popular project actually requiring that you didn't set noEmit: true in your tsconfig, but my memory is failing me as to what that is (was it ts-node? a webpack loader? it's been too long ugh)

@weswigham
Copy link
Member

For anything actually running TS in node at runtime, I think it'd have to be node16 + noEmit allowing direct .ts references.

@weswigham
Copy link
Member

weswigham commented Aug 3, 2022

I do have some concerns with the approach of enabling .ts extensions without its own flag. Maybe over time noEmit could imply that, the same way that checkJs implied allowJs. But maybe my concerns are not well-founded.

Maybe to dovetail with #50133, we should have a allowedNonJsRuntimeExtensions: [".css", ".html", ".es", ".ts"] sort of option.

@andrewbranch
Copy link
Member Author

@jakebailey I believe you’re thinking of ts-loader, but only in “full mode” as opposed to transpileOnly, which has been in the back of my mind to look into, but basically everyone uses transpileOnly

@DanielRosenwasser .ts imports in Node proper would be enabled in nodenext under noEmit and/or whatever other options we pick to make it work in any other mode. We will have to fix the resolution bug I mentioned, which I think we should do soon even if we don’t do anything else soon.

@jakebailey
Copy link
Member

basically everyone uses transpileOnly

Not my last team... 😨 (I think for const enum inlining, and one weird dependency that is only a d.ts file and declares a const enum.)

@andrewbranch
Copy link
Member Author

I feel confident that either it will not matter at all or that the combined will and know-how of me and @johnnyreilly will be up to the task.

@wuzzeb
Copy link

wuzzeb commented Aug 4, 2022

As a library author, one thing which would be nice is to allow moduleResolution to be an array, which tells typescript that imports must be valid in all of the listed types. So this would be even more restrictive, as library authors we need to write imports in a way that all the module resolutions work and typescript could help by giving an error. So we could specify moduleResolution: ["conventional", "node16"] and have typescript verify that our imports are correct no matter if the consumer of our library uses conventional or node16.

There wouldn't really need to be any extra special logic in typescript, if moduleResolution is an array typescript would just process each import twice and error if they are not all equal.

@frank-dspeed
Copy link

frank-dspeed commented Aug 5, 2022

I guess its still time to mention TypeScript Should let the Host System do Specifier Lookup let me explain
The ECMAScript Standard Says Specifiers get Resolved on the Language Embedder Side

In Our Case this is TypeScript it self i my self Authored a Lot of Engines so Host Systems that Embedded ECMAScript

my Proposal is finally Merge Rollup or at last its algo under module Resolution rollup into typescript

Rollup is a loader / bundler and a universal ModuleSystem all in One.
Rollup is a collection of hooks and processes to do Specifier lookup and loading then creates a Expression Tree
that is abstract so Universal.

All Existing Host Engines do offer Ways to Manipulate Specifier Resolution

  • ServiceWorker (Browser) Request Interception rerouting.
  • NodeJS Loader Hooks
  • GraalJS Context Filesystem Modification or shim
  • Deno Bundle everything always before load and this way Configure Specifier Lookup

Advanced

rollup uses Acorn as AST which is more flexible i suggest changing the Typescript AST with the Acorn one for better Interop in general while there exist Shims for both AST's to work with each other.

the missing parts could get created as Acorn plugin

More Big Proposal

Do not Integrate a Plugin system for specifier lookup kick it all out and go back to only support moduleResolution Classic
and then we do in user-space offer Fuse or something else to provide the right file structure.

also supporting a lookup on package-manager level would be guess able. as mentioned the lockfiles do container specifier resolution at last for the complet module system of the whole project.

I guess Classic + Files containing References or .d.ts only exports is the right way to go. This way we have the resolved parts like with a package-lock.json.

@romulocintra
Copy link

Hi @andrewbranch this really interesting proposal, we have a recurring meeting at TC39 focusing on Tools with folks from(Deno, Parcel, etc..). It would be nice to bring this to the discussion. Send me a DM or fill the form in case you want to participate. :)

@Jack-Works
Copy link
Contributor

about DTS resolution, can we emit code like this to let all declaration brings their own resolution mode instead of inheriting from the project?

// my-library/dist/index.d.ts
export { util } from './utils/index.js' asserts { "resolution-mode": [...] }

@robpalme
Copy link

robpalme commented Aug 9, 2022

Thanks for taking the time to study this deeply. Module resolution is fundamental to the entire web tooling stack so getting it right is valuable and appreciated.

Initial Reaction

We know this is a tough problem space from experience. Inside Bloomberg we have our own runtime environment (everyone seems to be doing it these days) with its own custom resolution semantics. As surmised in Background, just like all the other non-Node runtimes we chose to use "moduleResolution": "node" to approximate our system's resolver. It is not a perfect fit because our resolver is more browser-like and does not do a recursive walk of node_modules directories and does not feature any package.json files. Therefore the proposed "minimal" mode is a closer approximation to our resolver and we would make use of it 👍

Today we handle resolution of bare-specifiers using "paths" to enumerate the full set that can legally be imported by the project. In our system, each project's tsconfig is tool-generated, so this is easy. I'm expecting "paths" will continue to compose nicely with the new modes.

Explicitness as a Guiding Principle

The overall direction of this proposal is to permit explicitness in source code as a way of ensuring correctness. I think this is our North Star and we can use it to resolve some of the consequential issues.

Explicitness matters most obviously in the runtime. As the web platform evolves to allow more file formats to be directly imported (JS, CSS, JSON, Wasm, et al) extensionless imports become a hazard. It's hard to know what import x from "./foo" will mean if you have a directory containing:

- foo.js
- foo.css
- foo.json
- foo.wasm
- foo.<whatever is introduced next>

Bundlers answer this question by implementing priority order. require() also does this. But for the wider web platform, there is no universal precedence ordering of file types. Browsers won't answer this question either; they use explicit mime types and regardless, would not tolerate resolution based on multiple round-trips.

So the preference for our codebases is to avoid betting on a specific bundler's convention and instead sidestep this hazard by ensuring that relative specifiers in source code always use the full source code filename explicitly including its extension.

Importing Declaration Files

in these modes, I suggest that it be legal to write an import type of a module specifier ending in .d.ts.

This is an appreciated step forwards towards permitting and encouraging explicitness 👍

Low priority suggestion: For relative specifiers in "minimal" mode, emitting these extensions in Declaration Emit could also be a useful follow-on step. It's all more clues for the reader and safeguards against the introduction of new extensions in future.

Emitting relative module specifiers ending in .ts

The proposal states that resolution of .ts files will be supported by minimal, but emitting will not. Supporting resolution is highly appreciated and represents another crucial step forwards towards explicitness 👍

However, the ban on emit seems unnecessarily restrictive and is the main thing we would seek to change 🙏

Our internal platform will resolve whatever you point it at. Just like the web. So it would be fine to deploy .ts files and have them import each other. This is primarily desirable from a user's mental model - because the source graph is then coherent, rather than forcing users to reference invisible predicted file names. I agree that writing anything else (e.g. .js) is "nonsensical for the context".

Beyond developer comprehension, using .ts references will also allow us to reduce tooling complexity by avoiding file & specifier remapping. In our system we ban the small number of TypeScript concepts that emit runtime code (enums, parameter properties, etc). So as a concrete example of tooling simplicity, you could imagine our runtime having a development-time file transform loader that makes the TypeScript code executable solely by replacing types with whitespace. This would make sourcemaps redundant.

The counter arguments in the proposal (see "I think much of the demand we’ve heard so far for being able to use .ts-suffixed module specifiers has been misplaced") don't seem to apply here. This use-case is not about working around problems with bundlers. And it's not about shipping the code to multiple environments. It's purely about having freedom.

If TypeScript enforces the noEmit restriction, we can still workaround it in a few ways. But the options are not ideal:

  • Option 1 - Fork TypeScript
    • ☹ incurs a small maintenance risk/cost
    • Our pride in shipping 100% Genuine TypeScript™ so far is mildly damaged
  • Option 2 - Switch to a third-party TS->JS converter e.g. esbuild or swc
    • ☹ risks divergence from tsc
    • It feels weird to make a big change to our tooling stack for such a small request
    • We love having coverage of TypeScript JS emit in our system. It allows us to study the diffs across our entire codebase whenever there is a Beta release, which lets us feedback issues early such as Bloomberg feedback on TypeScript 4.5 Beta #46382

Conclusion

To sum up, the core proposal is awesome and includes two key features we value highly:

👍 "minimal" resolution
👍 the ability to reference .ts files

The request to allow emit of .ts specifiers is not a showstopper and could be debated/delivered as a follow-on proposal.

@RyanCavanaugh
Copy link
Member

RyanCavanaugh commented Aug 9, 2022

Our internal platform will resolve whatever you point it at. Just like the web. So it would be fine to deploy .ts files and have them import each other

@robpalme Just so I'm clear -- your runtime is capable of handling a situation where there are no .ts files on disk, but two JS files are allowed to refer to each other using their original .ts filenames? e.g.

// a.js
import * as b from "./b.ts";

// b.js
export { }

How does it know if there was ever a .ts file in the first place? Or is it just totally agnostic?

Edit: Fixed a critical typo in this example (b.js was previously b.ts)

@robpalme
Copy link

robpalme commented Aug 9, 2022

@RyanCavanaugh I'm describing a different case. Meaning if there is a *.anything on disk, then you can use import foo from "*.anything". Meaning the file extension is not special. It's just files referencing files.

@frank-dspeed
Copy link

@robpalme did you consider using references the tripple slash once to reference files from files? i do that today and it is a win for me. Google tripple slash typescript If you are not aware of that.

for bundling i simply translate the tripple slash references to imports

@RyanCavanaugh
Copy link
Member

@robpalme and I discussed the emit vs .ts imports thing offline. TL;DR -- the scenario he describes above would still be well-supported.

The nuance here is that we would still intend to block .ts -> .js emit via tsc in the case where we see .ts imports, since it's unreasonable to think that a runtime could see a .ts -> .ts import in files named .js and treat that correctly (it's not clear how, and especially unclear why, anyone would implement that). However, API-level operations like ts.transpileModule would not be gated off -- tools would still be free to use the TS API to downlevel/type-strip arbitrary content regardless of the file extensions in import statements.

@frank-dspeed
Copy link

@robpalme do you know maybe anyway how we can push the adoption of tc39 type annotation proposal https://github.com/tc39/proposal-type-annotations when we would get that landed even if it would be only babel

that would open up the end of the resolve story at all. as we can simply author then in .js

@Jarred-Sumner
Copy link

Thank you @robpalme, we will try that.

@hlovdal
Copy link

hlovdal commented Nov 8, 2022

I would like to emphasize what @robpalme brought up earlier, explicitness.

Including resources WITH a file extension SHOULD be the standard. Please work towards making this possible.

Hiding the file extension adds a new layer of complexity and adds ambiguity. Code is much easier to reason about when there is a 1 to 1 mapping, e.g. #include "something.h" -> there is a file named exactly something.h and you can easily search for it.

All such hiding of parts of the resource name that the user interacts with hurts more than it helps (and don't get me started on Windows file explorer hiding file extensions).

The same applies for instance to Angular cli's unhelpful "fiendly" behaviour of automatically adding src/app in front of the argument you give when using the tool to generate new things. Until you're burned enough times your first attempt to run the cli tool in the most intuitive and natural way, e.g. ng generate component src/app/feature/todo always fails because you then end up with src/app/src/app/feature/todo/todo.component.* files which obviously was not what was intended.

Or LaTeX.

# Is the logo image used anymore?
ack -l logo.png *.tex
# Well the above query will not tell because maybe there still is an \includegraphics{logo} somewhere...

Or similarly the related lack of command line argument consistency for npm

npm start     # Runs "start" entry in scripts in package.json
npm mystart   # Fails to run "mystart" entry in scripts in package.json

If npm developers have had any decency in their command line API design the start sub command should newer have existed and instead everything should run through the generic run sub command, e.g. consistently

npm run start
npm run mystart

If typing npm run start is too much work then you can write your own npmstart.sh, start.rex or whatever shortcut in your favourite scripting language.

From Joshua Bloch's excellent talk How to design a good API and why it matters (slides):

User of API should not be surprised by behavior

  • It's worth extra implementation effort
  • It's even worth reduced performance

Hiding file extensions will cause surprises and is a bad idea.

In C++ they switched from #include <iostream.h> to #include <iostream> but at least the corresponding file was also correspondingly renamed so that #include <iostream> maps to just /usr/include/c++/11/iostream and NOT /usr/include/c++/11/iostream.h. So that is a removal, not hiding; there is no file name translation magic involved here.

Explicitness and consistency is vastly more important than laziness and convenience.

@frank-dspeed

This comment was marked as off-topic.

@lemanschik

This comment was marked as off-topic.

@lemanschik

This comment was marked as off-topic.

@andrewbranch
Copy link
Member Author

Wanted to give a few updates here.

  1. The module resolution mode for bundlers, called conventional in this proposal and then subsequently hybrid, is implemented at --moduleResolution bundler (formerly known as hybrid) #51669 but will likely not be called hybrid after all. A discussion is ongoing in What should the new moduleResolution setting be called? #51714. I hope to settle on a name and merge the PR this week, which means it will ship in 5.0.
  2. We decided in a design meeting a few weeks ago to hold off on minimal for right now, because with its intentional lack of support for node_modules, the story around resolving typings for dependencies that you put in your app somewhere, whether in a folder called node_modules or vendor or anything else, felt very incomplete. A bit of discussion on this is at Can’t resolve @types when vendoring dependencies for browser ESM imports #50600. I’m still very interested in having a module resolution mode that’s appropriate for browser-native ESM, but I’m struggling to get much real-world feedback on how folks are approaching this. If you have a frontend app, especially written in TypeScript, especially with dependencies, that uses ESM in the browser without going through a bundler first (I know Vite emits ES modules, but it handles module resolution internally so it’s irrelevant to this scenario), I definitely want to hear from you! But for now, hybrid was the higher priority and minimal will not land in 5.0.
  3. I plan to add to and update our module documentation, which is pretty scattered/incomplete/stale right now. If you have any current questions or points of confusion about modules and TypeScript, or if you used to be confused but something in particular made it click for you, please tell me about it in What’s confusing about modules? #51876.

@IanVS
Copy link

IanVS commented Aug 22, 2023

Should this be closed now, given the Bundler resolution option? Or is there still more to be done?

@andrewbranch
Copy link
Member Author

I was keeping it open because we never shipped anything for minimal, which is blocked by the design problem of #50600. We’ve received very little feedback in the meantime that people need it though, so there are no immediate plans to make anything new.

@MicahZoltu
Copy link
Contributor

If you have a frontend app, especially written in TypeScript, especially with dependencies, that uses ESM in the browser without going through a bundler first (I know Vite emits ES modules, but it handles module resolution internally so it’s irrelevant to this scenario), I definitely want to hear from you!

I don't know how I missed this request, but I build all of my frontend apps with ES modules and no bundler with TypeScript.

Here is the template I use for all of my apps:
https://github.com/Zoltu/preact-es2015-template

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
In Discussion Not yet reached consensus Suggestion An idea for TypeScript
Projects
None yet
Development

No branches or pull requests