build(drivers/framework): Fix type generation for ESM and add exports field

[tylerbutler]

This PR updates the majority of packages in packages/drivers and packages/framework to generate proper types in ESM builds and adds an exports field. The packages were already building using moduleResolution: node16 by virtue of inheriting from the new shared configs, so no changes were needed there. They also have a test using arethetypeswrong that runs in CI.

I did NOT add an exports field to the odsp-driver package, because doing that right now breaks a lot of downstream compilation. We should be able to resolve that once more/all packages are using node16 module resolution.

[msfluid-bot]

⯅ @fluid-example/bundle-size-tests: +18 Bytes

Metric Name	Baseline Size	Compare Size	Size Diff
aqueduct.js	503.59 KB	503.6 KB	⯅ +4 Bytes
connectionState.js	680 Bytes	680 Bytes	■ No change
containerRuntime.js	291.51 KB	291.51 KB	■ No change
loader.js	187.22 KB	187.22 KB	⯅ +2 Bytes
map.js	99.02 KB	99.02 KB	⯅ +2 Bytes
matrix.js	198.33 KB	198.33 KB	⯅ +2 Bytes
odspDriver.js	111.61 KB	111.62 KB	⯅ +2 Bytes
odspPrefetchSnapshot.js	67.24 KB	67.25 KB	⯅ +2 Bytes
sharedString.js	216.16 KB	216.16 KB	⯅ +2 Bytes
sharedTree2.js	322.91 KB	322.91 KB	■ No change
Total Size	2.06 MB	2.06 MB	⯅ +18 Bytes

---

Baseline commit: 04a597c

Generated by 🚫 dangerJS against 7e1756d

[tylerbutler]

policy-check caught this missing dep

[jason-ha]

"types": "dist/index.d.ts",

no exports?

---

Refers to: packages/drivers/odsp-driver/package.json:16 in aa81da0. [](commit_id = aa81da0, deletion_comment = False)

[jason-ha]

I guess I am a little late to the party. Renaming doesn't feel like a robust build step. Is it not possible for compile step to output .d.mts?

[tylerbutler]

I guess I am a little late to the party. Renaming doesn't feel like a robust build step. Is it not possible for compile step to output .d.mts?

I don't love it either, but every time I tried to do it in tsc-multi I broke something else. See commentary at tylerbutler/tsc-multi#1 along with the tsc-multi patch I made to make it rewrite the imports.

[tylerbutler]

"types": "dist/index.d.ts",

no exports?

Refers to: packages/drivers/odsp-driver/package.json:16 in aa81da0. [](commit_id = aa81da0, deletion_comment = False)

Assuming you're referring to odsp-driver, good catch. I updated the PR description to explain that:

I did NOT add an exports field to the odsp-driver package, because doing that right now breaks a lot of downstream compilation. We should be able to resolve that once more/all packages are using node16 module resolution.

[jason-ha]

I guess I am a little late to the party. Renaming doesn't feel like a robust build step. Is it not possible for compile step to output .d.mts?

I don't love it either, but every time I tried to do it in tsc-multi I broke something else. See commentary at tylerbutler/tsc-multi#1 along with the tsc-multi patch I made to make it rewrite the imports.

I looked up more information on this. Surely there is a tsc option. No, there surely is not. And the reason seems to be that typescript is formally against dual emit.
From microsoft/TypeScript#49462 (comment) and on they point to https://nodejs.org/api/packages.html#dual-package-hazard. The solution to avoid seems to be use a wrapper for the secondary. Options seem best called out microsoft/TypeScript#49462 (comment).

Are we making sure that all of our clients understand fullstack and we are only getting ESM or CommonJS? The fact that we run into issues downstream internally is concerning.

If our long term plan is to drop CommonJS, then okay.

I think there is another option for the rename issue. We inject a package.json with { "type": "module" } in lib folder. Then all of the .d.ts files (and any .js files (none, right?)) will be ESM.

[jason-ha]

I did NOT add an exports field to the odsp-driver package, because doing that right now breaks a lot of downstream compilation. We should be able to resolve that once more/all packages are using node16 module resolution.

That doesn't seem quite right. Shouldn't those fallback?

[edit] Ah, well, perhaps the problem is that those folks downstream are using illegal imports. It isn't the module resolution in here that matters but that exports is defined and limited.?

[jason-ha]

Please do edit the description before merging to be more about the commit than the PR.
Call out the changes that do land in this commit (versus elsewhere).
Also recommend noting why odsp-driver doesn't set exports.

[tylerbutler]

[edit] Ah, well, perhaps the problem is that those folks downstream are using illegal imports. It isn't the module resolution in here that matters but that exports is defined and limited.?

That's true for internal usage and is the primary blocker. However, we did release internal 7.3 with an expoerts field that pointed to non-existent types for ESM, and that definitely broke some people. Now that we have are-the-type-wrong tests in place I am more confident that our packages will work correctly in both node10 and node16 modes when we do enable exports in odsp-driver.

[tylerbutler]

I think there is another option for the rename issue. We inject a package.json with { "type": "module" } in lib folder. Then all of the .d.ts files (and any .js files (none, right?)) will be ESM.

From an earlier internal discussion:

There are lots of articles online about dual-emit with TS. Most of them are either outdated or wrong. For example, a common suggestion is to put a dummy package.json file in your output dirs (i.e. dist and lib) with the type set correctly. But this has problems with import maps (see this comment in the TS issue for more details), not to mention package.json files in random places confuse the crap out of fluid-build (fixable no doubt).

The best solution I have found is to manually do what folks are suggesting TS do. That is, build twice, once with type: module and once with type: commonjs, then manually (a) change the file extensions to .cjs or .mjs as appropriate and (b) update the imports in each file to use the different extension. It's not pretty.

Relevant bit from the other comment:

This isn't an approach I would recommend. At one point I was looking into using "stub" package.json files to set { "type": "module" } in a specific directory to support dual emit using the TypeScript API. At first glance, it seems like this might work as long as you only ever use relative imports, or import from packages in your node_modules. However, if you use an import map and # imports to other locations in the root of your package, those imports will no longer work.

[jason-ha]

[edit] Ah, well, perhaps the problem is that those folks downstream are using illegal imports. It isn't the module resolution in here that matters but that exports is defined and limited.?

That's true for internal usage and is the primary blocker. However, we did release internal 7.3 with an expoerts field that pointed to non-existent types for ESM, and that definitely broke some people. Now that we have are-the-type-wrong tests in place I am more confident that our packages will work correctly in both node10 and node16 modes when we do enable exports in odsp-client.

Does this sufficiently address any TS dependees with moduleResolution set to node? They will still see package.json module and types referencing the CommonJS declaration file (dist/index.d.ts).? Or will the presence of lib/index.d.mts supercede types?
If the former, they aren't fixed. Either way we should probably remove module.

[jason-ha]

That comment in the commit is not correct from what I understand.
It is not use of node16 resolution that makes the difference for downstream, it is the limiting of imports to those exported that matter.

[tylerbutler]

Does this sufficiently address any TS dependees with moduleResolution set to node? They will still see package.json module and types referencing the CommonJS declaration file (dist/index.d.ts).?

Yes, it should work for people using node10 in TypeScript. What node10 users won't be able to use is the non-default exports, like /beta. But node10 is very permissive, just like node is with commonjs.

Keep in mind that the module resolution typescript uses has no bearing on what node does at runtime. If you take code compiled in typescript moduleResolution node10 and run the JS output in node 18, node will resolve according to node16 rules. It might work. It might not. This is one of the things I was setting out to fix when I started working in this area. We produce ESM but no one can actually use it, and if they tried to use it in node, it wasn't going to work. Compiling using node16 is the only supported way to get Typescript to help prevent you from shipping bad imports. Thus, while I can confidently say our packages will work with node10, I am also reasonably confident anyone compiling for LTS+ versions of node (i.e. not for a bundler) with that moduleResolution will produce code that won't run in node.

[tylerbutler]

That comment in the commit is not correct from what I understand.
It is not use of node16 resolution that makes the difference for downstream, it is the limiting of imports to those exported that matter.

Sort of. If a downstream consumer is using node10, then they can continue to dig into the internals of a package. Even if we add an exports field, it's not used in node10 as far as I can tell. This seems to corroborate that: https://www.typescriptlang.org/tsconfig#resolvePackageJsonExports

[jason-ha]

That comment in the commit is not correct from what I understand.
It is not use of node16 resolution that makes the difference for downstream, it is the limiting of imports to those exported that matter.

Sort of. If a downstream consumer is using node10, then they can continue to dig into the internals of a package. Even if we add an exports field, it's not used in node10 as far as I can tell. This seems to corroborate that: https://www.typescriptlang.org/tsconfig#resolvePackageJsonExports

The comment says downstream need to use node16 (or nodenext) moduleResolution before odsp-driver can add exports specification. But if node10 doesn't use exports then why would it matter? Downstream must be paying attention to exports for them to matter. Using node16+ isn't going to resolve the issues (which I guess are likely "illegal" imports).

[jason-ha]

Keep in mind that the module resolution typescript uses has no bearing on what node does at runtime. If you take code compiled in typescript moduleResolution node10 and run the JS output in node 18, node will resolve according to node16 rules. It might work. It might not. This is one of the things I was setting out to fix when I started working in this area. We produce ESM but no one can actually use it, and if they tried to use it in node, it wasn't going to work. Compiling using node16 is the only supported way to get Typescript to help prevent you from shipping bad imports. Thus, while I can confidently say our packages will work with node10, I am also reasonably confident anyone compiling for LTS+ versions of node (i.e. not for a bundler) with that moduleResolution will produce code that won't run in node.

I don't think you can simplify things like the first statement tries. Typescript is being very explicit about trying to match node as best it can. You quoted https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-format-detection in a Teams chat. TSC cannot know what the ultimate version of node is and what rules are at play, but

1. when not module = node16|nodenext: it can try to be agnostic (compatible) given all of the things it knows about or
2. otherwise: follow the very specific rules.
   Proper coordination of package.json meta info guides node to matching files. There really isn't much point without moduleResolution being node16|nodenext as they are the versions that support ESM. Plus, we only support node 18+; so, there isn't much point in thinking about anything node 10 or less. Even if a package doesn't emit ESM, it should still use moduleResolution=node16|nodenext.

The dual emit goes against explicit typescript recommendations in more than the dual hazard.
From Typescript Handbook Modules reference:

• node16 and nodenext are the only correct module options for all apps and libraries that are intended to run in Node.js v12 or later, whether they use ES modules or not.

We workaround with module=esnext and moduleResolution=node16 as we don't have "type": "module" in package.json. According to documentation notes that seems okay so long as we don't use import x = require("..."), which is easy enough.

Is there any reason to keep "module": in package.json files?