Use .js extensions for imports to so that source is ESM #1519

keller-mark · 2023-05-02T21:38:41Z

(This is a PR into the Zod PR)

Adds JS file extensions in all files to conform to TS and ESM requirements (has been discussed in TS repo issues for years)
- This unfortunately touches every JS file that performs a relative import
- Ideally, TypeScript could add the .js endings during its compilation step, but the TypeScript developers seem very against that - it seems like the reasoning is to keep the scope of TS to only 1) performing type checking and 2) stripping types from the file. It is odd because TS does perform JSX->JS transformation, though perhaps the reasoning there is that it has a 1-to-1 mapping to the JS syntax while updating import statements could change the behavior?
Updates the eslint config to enforce this
Updates documentation for internal developers in dev-docs
Adds publint to check package.json
Updates package.json files to reference the ESM and use type: module everywhere

Demo of documentation: http://vitessce-data.s3-website-us-east-1.amazonaws.com/docs/2023-05-11/512964ad/docs/upgrade-guide/

Checklist

Ensure PR works with all demos on the dev.vitessce.io homepage
Open (draft) PR's into vitessce-python and vitessce-r if this is a release PR
Documentation added or updated

ilan-gold · 2023-05-10T14:46:28Z

@keller-mark a few things:

This PR doesn't actually check for esm-validity, right (or that the packages can be imported via cdn)?
Does this have much to do with the zod PR besides the fact that that PR will go first?
Does @manzt have any suggestions here?
Are there any changes besides the docs and appending js to imports?

ilan-gold

Some initial comments.

sites/docs/docs/about.md

sites/docs/docs/tutorial-plugin-coordination-type.mdx

sites/docs/docs/tutorial-plugin-file-type.mdx

dev-docs/design-guidelines.md

keller-mark · 2023-05-10T14:55:11Z

packages/main/prod/package.json

+  "type": "module",
  "main": "src/index.js",
  "publishConfig": {
-    "main": "dist/index.min.mjs",
-    "browser": "dist/index.min.umd.js"
+    "main": "dist/index.min.js",
+    "module": "dist/index.min.js",
+    "exports": {
+      ".": {
+        "types": "./dist-tsc/index.d.ts",
+        "import": "./dist/index.min.js"
+      }
+    }
  },
  "files": [
+    "src",
    "dist",
-    "src"
+    "dist-tsc"


Related to #1374 - this PR corrects package.json files to use type: module and also removes references to UMD builds from package.json files. Those UMD builds are still in dist/ but this effectively hides them as a step towards removing them if there are no issues encountered.

As such, I removed the UMD cypress tests and added cypress tests with https://github.com/keller-mark/dynamic-importmap. I have been testing dynamic-importmap in other places: it works on observable with the existing ESM builds and on colab with anywidget

keller-mark · 2023-05-10T16:07:17Z

Thank you for the review! You are right about the importing .js files from packages being odd, I will work to remove that.

Originally, I think the direct within-package exports (import CloudIcon from '@material-ui/icons/Cloud' and import cloneDeep from 'lodash/cloneDeep') were to not only to ensure that the unrelated parts of those libraries were not included in the Vitessce bundle, but also to ensure that unrelated code did not need to be loaded by the dev server during development.

However I think all modern bundlers including Vite and Rollup now support tree-shaking of ESM named imports and are clever/performant enough that the development experience is unaffected. Rollup is even going to be able to tree-shake dynamic imports soon.

I guess this is my concern - how much are we changing here, not just in our own codebase but requirements that other codebases do certain things?

We are changing the import statements to conform to the ESM requirements here but our development tools like Vite can still load CommonJS or UMD libraries (albeit with the undesired .js extension for named imports with those). I think this is getting to be less of a concern going forward: in trying to address your above feedback, I found many of the libraries we are using have specific guidance about ESM imports:

As for the others:

colormap: colormap/colorScale.js is not re-exported from its index.js entrypoint
react-virtualized: their ESM is broken Bogus import in generated dist ESM file bvaughn/react-virtualized#1632
json2csv: their ESM is also broken Error module stream in ESM build zemirco/json2csv#539
import { getDataAndAria } from 'rc-tree/es/util.js';: here we are essentially using a private export - it is not re-exported from the main package entrypoint (similar to colormap case above)

…eller-mark/file-exts

sites/docs/docs/js-overview.md

dev-docs/design-guidelines.md

* Create zod schema for 1.0.16 * Move to separate package * Revert main package.json changes * Use builder functions * wip: previous versions * Upgrade functions * Initial tests * VitS as controlled component * Upgrade in <Vitessce/> * WIP: plugins registered via props (instead of window) * Update schemas * Joint file type schemas and via new plugin mechanism * Refactor plugin classes * Cleanup * Update * More cleanup * Cleanup register functions * Remove plugins file, use default coordination values from plugin objects * Comment * Use superRefine to check for deprecated coordination types * Revert * Fix tests * Extra imports * Try env.d.ts * Update * Update plugin examples * Linting for obsSets view * Comment * Allow ts in vite bundling config * Fix e2e tests * Update * Fix import * Allow empty props * Update * Remove ajv * Update * Update deps * Lint * Fix tests * Implement plugin classes with typescript * Add config uid property * use .length * Fix docs * Update plugin docs * Add config schema versioning docs * Add useCallback calls to obsSetsManager * Remove unnecessary fromEntries * Implement getSourceAndLoader function * Remove extra import * Eslint comment * Demo * Update dev-config-versioning.md * Update docusaurus.config.js * Lint * Make json schemas * Tsconfig * Lockfile * Update deploy script and readme * Simplify pnpm command * Update pnpm lock * Respond to PR feedback * Fix type issues, update docs * Refactor base plugins * Update * Pnpm install * Types in package.json * Fix tsconfig project references * Pnpm lock * Add dev-docs * Working types and linting * pnpm lock * Update * Fix test * Lint * Pnpm lock * React import * React import * More react imports * Update package.json exports in sub-packages, add new test for dynamic importmap * Add comment * Comment * Dev-docs * Add comment * Use dynamic-importmap * Try compressed-size-action * Update * Node options * env * Clean cmd * Revert unrelated github action change * Fix typo * Revert changed ObsSetsManager * Use .js extensions for imports to so that source is ESM (#1519) * Use .js * Comments * Revert webpack plugin * Update cypress e2e tests * Add/move dev-docs * Update design-guidelines.md * Fix meta-updater * Revert config * Update design-guidelines.md * Update README.md * Update design-guidelines.md * Update design-guidelines.md * Update design-guidelines.md * Update rollup.config.js * Revert unrelated change * WIP: clean up lodash, mui, uuid imports * More import fixes * De-duplicate imports * Dev-docs about js imports * Use publint * isEqual * Fix tests * Dev-docs * Dev-docs linked from main README * Demo * Feedback * Use same terminology in docs as code * Troubleshooting note * Upgrade guide

* Create zod schema for 1.0.16 * Move to separate package * Revert main package.json changes * Use builder functions * wip: previous versions * Upgrade functions * Initial tests * VitS as controlled component * Upgrade in <Vitessce/> * WIP: plugins registered via props (instead of window) * Update schemas * Joint file type schemas and via new plugin mechanism * Refactor plugin classes * Cleanup * Update * More cleanup * Cleanup register functions * Remove plugins file, use default coordination values from plugin objects * Comment * Use superRefine to check for deprecated coordination types * Revert * Fix tests * Extra imports * Try env.d.ts * Update * Update plugin examples * Linting for obsSets view * Comment * Allow ts in vite bundling config * Fix e2e tests * Update * Fix import * Allow empty props * Update * Remove ajv * Update * Update deps * Lint * Fix tests * Implement plugin classes with typescript * Add config uid property * use .length * Fix docs * Update plugin docs * Add config schema versioning docs * Add useCallback calls to obsSetsManager * Remove unnecessary fromEntries * Implement getSourceAndLoader function * Remove extra import * Eslint comment * Demo * Update dev-config-versioning.md * Update docusaurus.config.js * Lint * Make json schemas * Tsconfig * Lockfile * Update deploy script and readme * Simplify pnpm command * Update pnpm lock * Respond to PR feedback * Fix type issues, update docs * Refactor base plugins * Update * Pnpm install * Types in package.json * Fix tsconfig project references * Pnpm lock * Add dev-docs * Working types and linting * pnpm lock * Update * Fix test * Lint * Pnpm lock * React import * React import * More react imports * Update package.json exports in sub-packages, add new test for dynamic importmap * Add comment * Comment * Dev-docs * Add comment * Use dynamic-importmap * Try compressed-size-action * Update * Node options * env * Clean cmd * Use .js * Comments * Revert webpack plugin * Update cypress e2e tests * Add/move dev-docs * Update design-guidelines.md * Fix meta-updater * Revert config * Update design-guidelines.md * Update README.md * Update design-guidelines.md * Update design-guidelines.md * Update design-guidelines.md * Update rollup.config.js * Revert unrelated change * Revert unrelated github action change * WIP: clean up lodash, mui, uuid imports * More import fixes * De-duplicate imports * Dev-docs about js imports * Use publint * Fix typo * isEqual * Fix tests * Revert changed ObsSetsManager * Dev-docs * Dev-docs linked from main README * Demo * NextJS consumer test * NextJS consumer test * Changelog * README * Use .js extensions for imports to so that source is ESM (#1519) * Use .js * Comments * Revert webpack plugin * Update cypress e2e tests * Add/move dev-docs * Update design-guidelines.md * Fix meta-updater * Revert config * Update design-guidelines.md * Update README.md * Update design-guidelines.md * Update design-guidelines.md * Update design-guidelines.md * Update rollup.config.js * Revert unrelated change * WIP: clean up lodash, mui, uuid imports * More import fixes * De-duplicate imports * Dev-docs about js imports * Use publint * isEqual * Fix tests * Dev-docs * Dev-docs linked from main README * Demo * Feedback * Use same terminology in docs as code * Fix merge * Revert workspace change * Linting * Update Vis.js

keller-mark added 7 commits May 2, 2023 17:33

Use .js

0555624

Comments

10d0e89

Revert webpack plugin

16ed22a

Update cypress e2e tests

8882c6e

Add/move dev-docs

b3a0305

Update design-guidelines.md

6d4445a

Fix meta-updater

64f6010

keller-mark marked this pull request as ready for review May 4, 2023 17:58

Revert config

23a3282

keller-mark changed the title ~~Use .js~~ Use .js extensions for imports to so that source is ESM May 4, 2023

keller-mark requested a review from ilan-gold May 4, 2023 18:34

keller-mark added 6 commits May 10, 2023 08:43

Update design-guidelines.md

0b0b0f7

Update README.md

c343a02

Update design-guidelines.md

c41b292

Update design-guidelines.md

45936c2

Update design-guidelines.md

69bc990

Update rollup.config.js

26a569f

ilan-gold reviewed May 10, 2023

View reviewed changes

keller-mark commented May 10, 2023

View reviewed changes

keller-mark mentioned this pull request May 10, 2023

Bundle size check action #1526

Merged

3 tasks

Revert unrelated change

1021dd6

keller-mark mentioned this pull request May 10, 2023

Remove dead code from rollup config #1527

Merged

3 tasks

vitessce deleted a comment from github-actions bot May 10, 2023

Merge

e034cb4

keller-mark added 2 commits May 10, 2023 12:11

WIP: clean up lodash, mui, uuid imports

ad7e1f9

More import fixes

457f319

keller-mark mentioned this pull request May 10, 2023

Remove Showcase from About page #1528

Merged

3 tasks

De-duplicate imports

eef9a3a

keller-mark added 9 commits May 10, 2023 14:27

Dev-docs about js imports

2f637fd

Use publint

e68bf20

Merge

1f84cfe

isEqual

0e1f343

Fix tests

3e9602c

Merge branch 'keller-mark/zod' of github.com:vitessce/vitessce into k…

8c643a6

…eller-mark/file-exts

Dev-docs

226dc6d

Dev-docs linked from main README

512964a

Demo

573ad26

keller-mark mentioned this pull request May 15, 2023

Use .js extensions for imports to so that source is ESM keller-mark/pnpm-monorepo-example#25

Closed

keller-mark mentioned this pull request May 22, 2023

Visualization won't load in a production build (NextJS) #1536

Closed

keller-mark requested a review from ilan-gold May 22, 2023 23:10

ilan-gold approved these changes May 23, 2023

View reviewed changes

sites/docs/docs/js-overview.md Show resolved Hide resolved

dev-docs/design-guidelines.md Outdated Show resolved Hide resolved

Feedback

c5c7ddf

keller-mark merged commit 124f8c3 into keller-mark/zod May 23, 2023
5 checks passed

keller-mark deleted the keller-mark/file-exts branch May 23, 2023 14:32

keller-mark mentioned this pull request Jun 2, 2023

Fix MUI import #1575

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use .js extensions for imports to so that source is ESM #1519

Use .js extensions for imports to so that source is ESM #1519

keller-mark commented May 2, 2023 •

edited

ilan-gold commented May 10, 2023

ilan-gold left a comment

keller-mark May 10, 2023

keller-mark commented May 10, 2023 •

edited

Use .js extensions for imports to so that source is ESM #1519

Use .js extensions for imports to so that source is ESM #1519

Conversation

keller-mark commented May 2, 2023 • edited

Checklist

ilan-gold commented May 10, 2023

ilan-gold left a comment

Choose a reason for hiding this comment

keller-mark May 10, 2023

Choose a reason for hiding this comment

keller-mark commented May 10, 2023 • edited

keller-mark commented May 2, 2023 •

edited

keller-mark commented May 10, 2023 •

edited