Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import star works in dev but not in prod #15542

Closed
7 tasks done
schontz opened this issue Jan 8, 2024 · 18 comments · Fixed by #15619
Closed
7 tasks done

import star works in dev but not in prod #15542

schontz opened this issue Jan 8, 2024 · 18 comments · Fixed by #15619
Labels
inconsistency Inconsistency between dev & build p2-edge-case Bug, but has workaround or limited in scope (priority)

Comments

@schontz
Copy link

schontz commented Jan 8, 2024

Describe the bug

I am using an npm module that does not have TypeScript definitions. I imported it like so:

import * as rectClamp from 'rect-clamp' // ✅ dev ❌ prod

const clamped = rectClamp(...)

And it works fine in development. But when I build my code, I get a runtime error that the function is not defined. Changing the import like so works fine:

import rectClamp from 'rect-clamp' // ✅ dev ✅ prod

const clamped = rectClamp(...)

Reproduction

https://stackblitz.com/edit/vitejs-vite-kgbyd3?file=src%2Fmain.ts

Steps to reproduce

  1. Visit the repro
  2. In dev it prints Error: None
  3. npm run build
  4. Download the output and serve up dist locally
  5. It prints out Error: TypeError: y is not a function

System Info

On StackBlitz:


  System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 18.18.0 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.4.2 - /usr/local/bin/npm
    pnpm: 8.14.0 - /usr/local/bin/pnpm
  npmPackages:
    vite: ^5.0.8 => 5.0.11 


### Used Package Manager

npm

### Logs

<details>
<summary>StackBlitz `vite build --debug`</summary>

```shell
  vite:config no config file found. +0ms
  vite:config using resolved config: {
  vite:config   root: '/home/projects/vitejs-vite-kgbyd3',
  vite:config   base: '/',
  vite:config   mode: 'production',
  vite:config   configFile: undefined,
  vite:config   logLevel: undefined,
  vite:config   clearScreen: undefined,
  vite:config   optimizeDeps: {
  vite:config     disabled: 'build',
  vite:config     force: undefined,
  vite:config     esbuildOptions: { preserveSymlinks: false }
  vite:config   },
  vite:config   build: {
  vite:config     target: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari14' ],
  vite:config     cssTarget: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari14' ],
  vite:config     outDir: 'dist',
  vite:config     assetsDir: 'assets',
  vite:config     assetsInlineLimit: 4096,
  vite:config     cssCodeSplit: true,
  vite:config     sourcemap: false,
  vite:config     rollupOptions: {},
  vite:config     minify: 'esbuild',
  vite:config     terserOptions: {},
  vite:config     write: true,
  vite:config     emptyOutDir: null,
  vite:config     copyPublicDir: true,
  vite:config     manifest: false,
  vite:config     lib: false,
  vite:config     ssr: false,
  vite:config     ssrManifest: false,
  vite:config     ssrEmitAssets: false,
  vite:config     reportCompressedSize: true,
  vite:config     chunkSizeWarningLimit: 500,
  vite:config     watch: null,
  vite:config     commonjsOptions: { include: [Array], extensions: [Array] },
  vite:config     dynamicImportVarsOptions: { warnOnError: true, exclude: [Array] },
  vite:config     modulePreload: { polyfill: true },
  vite:config     cssMinify: true
  vite:config   },
  vite:config   configFileDependencies: [],
  vite:config   inlineConfig: {
  vite:config     root: undefined,
  vite:config     base: undefined,
  vite:config     mode: undefined,
  vite:config     configFile: undefined,
  vite:config     logLevel: undefined,
  vite:config     clearScreen: undefined,
  vite:config     optimizeDeps: { force: undefined },
  vite:config     build: {}
  vite:config   },
  vite:config   rawBase: '/',
  vite:config   resolve: {
  vite:config     mainFields: [ 'browser', 'module', 'jsnext:main', 'jsnext' ],
  vite:config     conditions: [],
  vite:config     extensions: [
  vite:config       '.mjs',  '.js',
  vite:config       '.mts',  '.ts',
  vite:config       '.jsx',  '.tsx',
  vite:config       '.json'
  vite:config     ],
  vite:config     dedupe: [],
  vite:config     preserveSymlinks: false,
  vite:config     alias: [ [Object], [Object] ]
  vite:config   },
  vite:config   publicDir: '/home/projects/vitejs-vite-kgbyd3/public',
  vite:config   cacheDir: '/home/projects/vitejs-vite-kgbyd3/node_modules/.vite',
  vite:config   command: 'build',
  vite:config   ssr: {
  vite:config     target: 'node',
  vite:config     optimizeDeps: { disabled: true, esbuildOptions: [Object] }
  vite:config   },
  vite:config   isWorker: false,
  vite:config   mainConfig: null,
  vite:config   isProduction: true,
  vite:config   plugins: [
  vite:config     'vite:build-metadata',
  vite:config     'vite:watch-package-data',
  vite:config     'vite:pre-alias',
  vite:config     'alias',
  vite:config     'vite:modulepreload-polyfill',
  vite:config     'vite:resolve',
  vite:config     'vite:html-inline-proxy',
  vite:config     'vite:css',
  vite:config     'vite:esbuild',
  vite:config     'vite:json',
  vite:config     'vite:wasm-helper',
  vite:config     'vite:worker',
  vite:config     'vite:asset',
  vite:config     'vite:wasm-fallback',
  vite:config     'vite:define',
  vite:config     'vite:css-post',
  vite:config     'vite:build-html',
  vite:config     'vite:worker-import-meta-url',
  vite:config     'vite:asset-import-meta-url',
  vite:config     'vite:force-systemjs-wrap-complete',
  vite:config     'commonjs',
  vite:config     'vite:data-uri',
  vite:config     'vite:dynamic-import-vars',
  vite:config     'vite:import-glob',
  vite:config     'vite:build-import-analysis',
  vite:config     'vite:esbuild-transpile',
  vite:config     'vite:terser',
  vite:config     'vite:reporter',
  vite:config     'vite:load-fallback'
  vite:config   ],
  vite:config   css: { lightningcss: undefined },
  vite:config   esbuild: { jsxDev: false },
  vite:config   server: {
  vite:config     preTransformRequests: true,
  vite:config     sourcemapIgnoreList: [Function: isInNodeModules$1],
  vite:config     middlewareMode: false,
  vite:config     fs: {
  vite:config       strict: true,
  vite:config       allow: [Array],
  vite:config       deny: [Array],
  vite:config       cachedChecks: false
  vite:config     }
  vite:config   },
  vite:config   preview: {
  vite:config     port: undefined,
  vite:config     strictPort: undefined,
  vite:config     host: undefined,
  vite:config     https: undefined,
  vite:config     open: undefined,
  vite:config     proxy: undefined,
  vite:config     cors: undefined,
  vite:config     headers: undefined
  vite:config   },
  vite:config   envDir: '/home/projects/vitejs-vite-kgbyd3',
  vite:config   env: { BASE_URL: '/', MODE: 'production', DEV: false, PROD: true },
  vite:config   assetsInclude: [Function: assetsInclude],
  vite:config   logger: {
  vite:config     hasWarned: false,
  vite:config     info: [Function: info],
  vite:config     warn: [Function: warn],
  vite:config     warnOnce: [Function: warnOnce],
  vite:config     error: [Function: error],
  vite:config     clearScreen: [Function: clearScreen],
  vite:config     hasErrorLogged: [Function: hasErrorLogged]
  vite:config   },
  vite:config   packageCache: Map(1) {
  vite:config     'fnpd_/home/projects/vitejs-vite-kgbyd3' => {
  vite:config       dir: '/home/projects/vitejs-vite-kgbyd3',
  vite:config       data: [Object],
  vite:config       hasSideEffects: [Function: hasSideEffects],
  vite:config       webResolvedImports: {},
  vite:config       nodeResolvedImports: {},
  vite:config       setResolvedCache: [Function: setResolvedCache],
  vite:config       getResolvedCache: [Function: getResolvedCache]
  vite:config     },
  vite:config     set: [Function (anonymous)]
  vite:config   },
  vite:config   createResolver: [Function: createResolver],
  vite:config   worker: { format: 'iife', plugins: '() => plugins', rollupOptions: {} },
  vite:config   appType: 'spa',
  vite:config   experimental: { importGlobRestoreExtension: false, hmrPartialAccept: false },
  vite:config   getSortedPlugins: [Function: getSortedPlugins],
  vite:config   getSortedPluginHooks: [Function: getSortedPluginHooks]
  vite:config } +5ms
vite v5.0.11 building for production...
src/main.ts (13:21) Cannot call a namespace ("rectClampBroken").
✓ 9 modules transformed.
dist/index.html                 0.46 kB │ gzip: 0.29 kB
dist/assets/index-s-MxoWxy.css  1.21 kB │ gzip: 0.63 kB
dist/assets/index-XVpmAzNt.js   4.01 kB │ gzip: 2.10 kB
✓ built in 284ms

Validations

Copy link

stackblitz bot commented Jan 8, 2024

Fix this issue in StackBlitz Codeflow Start a new pull request in StackBlitz Codeflow.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 10, 2024

It seems that esbuild handles the behavior of module.exports = fn differently from rollup and it seems to be related to vite rewriting import * as.

@bluwy bluwy added p2-edge-case Bug, but has workaround or limited in scope (priority) inconsistency Inconsistency between dev & build and removed pending triage labels Jan 10, 2024
@bluwy
Copy link
Member

bluwy commented Jan 10, 2024

I think that

import * as rectClamp from 'rect-clamp' // ✅ dev ❌ prod

const clamped = rectClamp(...)

Should have not worked in dev. If you import star rectClamp should be an object, and executing as a function shouldn't work. I wonder if it's because Vite is misapplying the interop from esbuild.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 10, 2024

I am confused why vite needs to perform the following operations during dev.

import * as rectClamp from 'rect-clamp' 

// rewrite 
import _rectClamp from 'rect-clamp';
const rectClamp = _rectClamp;

const lines: string[] = [`import ${cjsModuleName} from "${url}"`]
importNames.forEach(({ importedName, localName }) => {
if (importedName === '*') {
lines.push(`const ${localName} = ${cjsModuleName}`)

Has this changed the meaning of the grammar?

@bluwy
Copy link
Member

bluwy commented Jan 10, 2024

The code you linked which points to the function should have a brief description why. Basically for CJS packages, because you don't know the named exports in advanced, esbuild would bundle everything as module.exports = { named: ..., default ... }.

However, we still want import { named } from 'cjs-lib' to work, so we apply an "interop" here. I'm guessing the interop we're applying for namespace imports isn't correct.

Checking react-clamp, they're assigning the function to module.exports directly, so I guess that's unexpected in the handling code which would then incorrectly handle it.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 10, 2024

what I mean to say is why don't we translate it as follows?

if (importedName === '*') {
  lines.push(`const ${localName} = {
    default: ${cjsModuleName}
  }`);
}

Instead of

if (importedName === '*') {
  lines.push(`const ${localName} = ${cjsModuleName}`);
}

When the properties are directly assigned to module.exports and use import *, rollup will only expose the default property value.

function _mergeNamespaces(n, m) {
    for (var i = 0; i < m.length; i++) {
        const e = m[i];
        if (typeof e !== 'string' && !Array.isArray(e)) { 
         // There is no iterable property in rectClamp
          for (const k in e) {
            if (k !== 'default' && !(k in n)) {
                const d = Object.getOwnPropertyDescriptor(e, k);
                if (d) {
                    Object.defineProperty(n, k, d.get ? d : {
                        enumerable: true,
                        get: () => e[k]
                    });
                }
            }
        } }
    }
    return Object.freeze(Object.defineProperty(n, Symbol.toStringTag, { value: 'Module' }));
}

function getDefaultExportFromCjs (x) {
	return x && x.__esModule && Object.prototype.hasOwnProperty.call(x, 'default') ? x['default'] : x;
}

_mergeNamespaces({
    __proto__: null,
    default: getDefaultExportFromCjs(rectClamp)
}, [rectClamp])

If the default behavior is not modified, I think vite may need to add a layer of compatibility in the build-import-analysis built-in plugin to synchronize with rollup's behavior.

@bluwy
Copy link
Member

bluwy commented Jan 10, 2024

It's because the result of module.exports could be an object like { default: ... } and that can be assigned to the namespaced specifier directly. In this case, it's not an object but a function which trips Vite up.

I'm guessing the fix is to check if it has __esModule, and if true we can return the object directly, and if false we need to wrap with a { default: ... }? I'm not really sure yet the exact conditions when we should wrap it.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 10, 2024

How about keeping in sync with rollup?

_mergeNamespaces({
    __proto__: null,
    default: getDefaultExportFromCjs(rectClamp)
}, [rectClamp])

or the behavior of Namespace Imports can be consistent with the handling of Dynamic Import, which I think it's not bad either. @bluwy

str.overwrite(
expStart,
expEnd,
`import('${rewrittenUrl}').then(m => m.default && m.default.__esModule ? m.default : ({ ...m.default, default: m.default }))` +
getLineBreaks(exp),
{ contentOnly: true },
)

Change the interactive logic of CJS to the following.

if (importedName === '*') {
  lines.push(`const ${localName} = ${cjsModuleName} && ${cjsModuleName}.__esModule ? ${cjsModuleName} : ({ ...${cjsModuleName}, default: ${cjsModuleName} })`)
}

@schontz
Copy link
Author

schontz commented Jan 12, 2024

Keeping in sync with what rollup does seems like the safest approach.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 12, 2024

I am thinking about a question: why doesn't Vite leverage the ability of esbuild to transpile CommonJS into ESM during the build phase (perhaps it can also reuse pre-built artifacts)? Wouldn't this make it easier to smooth out inconsistencies in handling CommonJS artifacts between dev and build phases? 🤔

@patak-dev
Copy link
Member

I am thinking about a question: why doesn't Vite leverage the ability of esbuild to transpile CommonJS into ESM during the build phase (perhaps it can also reuse pre-built artifacts)? Wouldn't this make it easier to smooth out inconsistencies in handling CommonJS artifacts between dev and build phases? 🤔

This is what we did in Vite 3. We had experimental support for pre-bundling with esbuild during build these past years, but we decided to remove the feature. See the rationale here:

The tl;dr; is that we are going to try to go in the other way around to reduce inconsistencies, with Rolldown (that is compatible with rollup) replacing esbuild during dev for pre-bundling.

@bluwy
Copy link
Member

bluwy commented Jan 15, 2024

How about keeping in sync with rollup?

Sorry for not following this up. Been swamped 😢 Personally I'd go with your second option and not syncing with Rollup, unless the mergeNamespaces code can be inlined to the module, mainly to reduce having another network request.

Plus, the interop code is mainly to be compatible with esbuild's output. The rollup code is meant to be used on the raw files directly I think? Esbuild's output already partially handles CJS, so we can add a bit more code to make the compatibility work.

Ultimately, would the final interop code be something like this?

const ${localName} = typeof cjsModuleName === 'object' : cjsModuleName : { default: ${cjsModuleName} }

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 16, 2024

Thank you for your reply!

Sorry for not following this up. Been swamped 😢 Personally I'd go with your second option and not syncing with Rollup, unless the mergeNamespaces code can be inlined to the module, mainly to reduce having another network request.

I agree, the logic implemented by the helper code is not complicated, and directly injecting it into the original module does not seem to create a significant burden.

Ultimately, would the final interop code be something like this?

This problem is a bit tricky, I personally think that the output forms of namespace import and dynamic import should be consistent.

A module namespace object is an object that describes all exports from a module. It is a static object that is created when the module is evaluated. There are two ways to access the module namespace object of a module: through a namespace import (import * as name from moduleName), or through the fulfillment value of a dynamic import.

In other words, we may also need to support the following equation.

import * as a from "demo";
import("demo").then(b => {
  // true
  console.log(a === b);
})

I am not sure if what you are expressing means that satisfying the output of the development phase is sufficient as a subset of the output of the build phase.

@bluwy
Copy link
Member

bluwy commented Jan 16, 2024

In other words, we may also need to support the following equation.

I think if we can ensure that the shape of namespace imports and dynamic imports are the same, then it should be a good start. We don't have to make sure they're strictly equal. I guess reading about this more:

Change the interactive logic of CJS to the following.

if (importedName === '*') {
  lines.push(`const ${localName} = ${cjsModuleName} && ${cjsModuleName}.__esModule ? ${cjsModuleName} : ({ ...${cjsModuleName}, default: ${cjsModuleName} })`)
}

This one sounds like the best fix for me if it also fixes the issue.

I am not sure if what you are expressing means that satisfying the output of the development phase is sufficient as a subset of the output of the build phase.

I mean the interop code we're dealing with is mainly dev exclusive only. Build has its own different path. In dev, we're using esbuild to prebundle, so we should adapt to its output and make it consistent to how it'll work in build. Build doesn't use the prebundled output anymore, especially with #15184.

We should still implement a fix that makes dev and build consistent, but we don't have to copy over what build does completely to achieve the consistency.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 16, 2024

This one sounds like the best fix for me if it also fixes the issue.

Yeah, originally I was considering directly using the output behavior of dynamic import. However, I explained the reasons in the PR that the output behavior of dynamic import is very similar to rollup's implementation, but there are still boundary behaviors that make it inconsistent(e.g. only export Array).

// module main
module.exports = [1, 3, 4]

// index.js
import("main").then((dynamicImport) => {
  /**
   * !!! Bad Case !!!
   * dev output: { 0: 1, 1: 3, 2: 4, default: [1, 3, 4] }
   * build output: { default: [1, 3, 4] }
   */
   console.log(dynamicImport)
})

I think if we can ensure that the shape of namespace imports and dynamic imports are the same, then it should be a good start. We don't have to make sure they're strictly equal.

Yes, referencing equality in strict mode is still an issue. In fact, I wanted to discuss with you yesterday whether support for this is needed. If necessary, I feel that I can only create a cache layer to ensure that the references loaded and obtained in other modules remain consistent. I seem to have not thought of a better way to handle the issue of inconsistent citations.

@bluwy
Copy link
Member

bluwy commented Jan 16, 2024

Interesting. I guess we can add additional code to prevent that from happening. Even though it'll be very similar to Rollup's implementation, I think my main concern is that #15600 is completely porting Rollup's implementation into Vite. We can slowly adjust our interop code where needed, but I don't think we need to go that far to be exactly compatible, e.g. the Object.freeze stuff.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 16, 2024

I understand what you mean, so let's make it compatible through the following logic. @bluwy

const translate = (module) => {
  if (Array.isArray(module) || typeof module === 'string') {
    return {
      default: module
    }
  }
  if (module && module.__esModule) {
    return module;
  }
  return ({ ...module, default: module });
} 

@patak-dev
Copy link
Member

If we do that, could we avoid the helper and new virtual module? Even if it is a bit more verbose, it should still be fine.

@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
inconsistency Inconsistency between dev & build p2-edge-case Bug, but has workaround or limited in scope (priority)
Projects
None yet
4 participants