Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reached heap limit when building since v4.2 #13638

Closed
7 tasks done
dd-jonas opened this issue Jun 26, 2023 · 6 comments · Fixed by #13663
Closed
7 tasks done

Reached heap limit when building since v4.2 #13638

dd-jonas opened this issue Jun 26, 2023 · 6 comments · Fixed by #13663
Labels
p4-important Violate documented behavior or significantly improves performance (priority) performance Performance related enhancement

Comments

@dd-jonas
Copy link
Contributor

dd-jonas commented Jun 26, 2023

Describe the bug

Update: Looks like the sourcemap CLI option broke, no performance issue. See comments below.


Reopening this as it is still not solved. I don't know how to create a reproduction without sharing my whole codebase because I don't know what causes this. I'm open for suggestions though.

See comments below, looks like a performance issue introduced in 4.2.0.


I noticed since upgrading to 4.2 that my builds are failing on CI (self hosted runner for GH Actions, 4GB RAM). I'm running vite build --mode testing --sourcemap false with a React project. Since I'm not building source maps, I think it's unusual to reach the heap limit so easily, although I'm no expert on this. The build transforms 13702 modules, most of which are from @mui/icons-material. Even when removing this dependency which lowers the module count to around 3000, I still reach the heap limit. On 4.1.5 I don't have any issues with the exact same codebase.

Has there been any change since 4.1 that could have impacted this? I did notice a bump from rollup 3.10 to 3.20, but can't tell from the changelog what the impact is. Locally it builds just fine.

I haven't found anything in #2433 that helps me.

Last tested on 4.3.9

<--- Last few GCs --->

[1249209:0x562bf00]    74516 ms: Scavenge 1908.7 (1986.9) -> 1908.4 (1997.6) MB, 14.6 / 0.0 ms  (average mu = 0.605, current mu = 0.406) allocation failure; 
[1249209:0x562bf00]    74538 ms: Scavenge 1915.1 (1997.6) -> 1915.3 (1997.9) MB, 19.5 / 0.0 ms  (average mu = 0.605, current mu = 0.406) allocation failure; 
[1249209:0x562bf00]    75381 ms: Scavenge 1916.1 (1997.9) -> 1915.2 (2020.1) MB, 843.1 / 0.0 ms  (average mu = 0.605, current mu = 0.406) allocation failure; 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
(Omitted some lines for brevity)
Aborted (core dumped)
error Command failed with exit code 1[34]
Error: Process completed with exit code 134.

Reproduction

Not sure how to repro this without sharing my code

Steps to reproduce

vite build --mode testing --sourcemap false on the CI server.

System Info

System:
    OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (2) x64 DO-Premium-AMD
    Memory: 2.79 GB / 3.83 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 12.22.9 - /usr/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 8.5.1 - /usr/bin/npm

Used Package Manager

yarn

Logs

Click to expand!
2023-06-26T10:01:30.915Z vite:config bundled config file loaded in 636.75ms
2023-06-26T10:01:30.990Z vite:esbuild 63.34ms tsconfck init /home/github/my_path_here
2023-06-26T10:01:31.012Z vite:config using resolved config: {
  build: {
    target: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari14' ],
    cssTarget: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari14' ],
    outDir: 'dist',
    assetsDir: 'assets',
vite v4.3.9 building for testing...
    assetsInlineLimit: 4096,
    cssCodeSplit: true,
    sourcemap: 'false',
    rollupOptions: { onwarn: [Function: onwarn] },
    minify: 'esbuild',
    terserOptions: {},
    write: true,
    emptyOutDir: null,
    copyPublicDir: true,
    manifest: false,
    lib: false,
    ssr: false,
    ssrManifest: false,
    ssrEmitAssets: false,
    reportCompressedSize: true,
    chunkSizeWarningLimit: 500,
    watch: null,
    commonjsOptions: { include: [Array], extensions: [Array] },
    dynamicImportVarsOptions: { warnOnError: true, exclude: [Array] },
    modulePreload: { polyfill: true },
    cssMinify: true
  },
  plugins: [
    'vite:build-metadata',
    'vite:watch-package-data',
    'vite:pre-alias',
    'alias',
    'vite:react-babel',
    'vite:react-refresh',
    'vite-tsconfig-paths',
    'vite:modulepreload-polyfill',
    'vite:resolve',
    'vite:html-inline-proxy',
    'vite:css',
    'vite:esbuild',
    'vite:json',
    'vite:wasm-helper',
    'vite:worker',
    'vite:asset',
    'vite-plugin-svgr',
    'replace',
    'sentry-noop-plugin',
    'vite:wasm-fallback',
    'vite:define',
    'vite:css-post',
    'vite:build-html',
    'vite:worker-import-meta-url',
    'vite:asset-import-meta-url',
    'vite:force-systemjs-wrap-complete',
    'commonjs',
    'vite:data-uri',
    'vite:dynamic-import-vars',
    'vite:import-glob',
    'vite:build-import-analysis',
    'vite:esbuild-transpile',
    'vite:terser',
    'vite:reporter',
    'vite:load-fallback'
  ],
  server: {
    preTransformRequests: true,
    host: 'my.host.here',
    open: true,
    port: 8000,
    sourcemapIgnoreList: [Function: isInNodeModules],
    middlewareMode: false,
    fs: { strict: true, allow: [Array], deny: [Array] }
  },
  mode: 'testing',
  optimizeDeps: {
    disabled: 'build',
    force: undefined,
    include: [ 'react', 'react/jsx-dev-runtime' ],
    esbuildOptions: { preserveSymlinks: false }
  },
  esbuild: { jsxDev: false, jsx: 'automatic', jsxImportSource: undefined },
  resolve: {
    mainFields: [ 'module', 'jsnext:main', 'jsnext' ],
    browserField: true,
    conditions: [],
    extensions: [
      '.mjs',  '.js',
      '.mts',  '.ts',
      '.jsx',  '.tsx',
      '.json'
    ],
    dedupe: [ 'react', 'react-dom' ],
    preserveSymlinks: false,
    alias: [ [Object], [Object] ]
  },
  configFile: '/home/github/my_path_here/vite.config.ts',
  configFileDependencies: [
    '/home/github/my_path_here/vite.config.ts'
  ],
  inlineConfig: {
    root: undefined,
    base: undefined,
    mode: 'testing',
    configFile: undefined,
    logLevel: undefined,
    clearScreen: undefined,
    optimizeDeps: { force: undefined },
    build: { sourcemap: 'false' }
  },
  root: '/home/github/my_path_here',
  base: '/',
  rawBase: '/',
  publicDir: '/home/github/my_path_here/public',
  cacheDir: '/home/github/my_path_here/node_modules/.vite',
  command: 'build',
  ssr: {
    format: 'esm',
    target: 'node',
    optimizeDeps: { disabled: true, esbuildOptions: [Object] }
  },
  isWorker: false,
  mainConfig: null,
  isProduction: true,
  preview: {
    port: undefined,
    strictPort: undefined,
    host: 'my.host.here',
    https: undefined,
    open: true,
    proxy: undefined,
    cors: undefined,
    headers: undefined
  },
  envDir: '/home/github/my_path_here',
  env: {
    VITE_APP_API_URL: 'http://my.url.here/api',
    BASE_URL: '/',
    MODE: 'testing',
    DEV: false,
    PROD: true
  },
  assetsInclude: [Function: assetsInclude],
  logger: {
    hasWarned: false,
    info: [Function: info],
    warn: [Function: warn],
    warnOnce: [Function: warnOnce],
    error: [Function: error],
    clearScreen: [Function: clearScreen],
    hasErrorLogged: [Function: hasErrorLogged]
  },
  packageCache: Map(1) {
    'fnpd_/home/github/my_path_here' => {
      dir: '/home/github/my_path_here',
      data: [Object],
      hasSideEffects: [Function: hasSideEffects],
      webResolvedImports: {},
      nodeResolvedImports: {},
      setResolvedCache: [Function: setResolvedCache],
      getResolvedCache: [Function: getResolvedCache]
    },
    set: [Function (anonymous)]
  },
  createResolver: [Function: createResolver],
  worker: {
    format: 'iife',
    plugins: [
      'vite:build-metadata',
      'vite:watch-package-data',
      'vite:pre-alias',
      'alias',
      'vite:modulepreload-polyfill',
      'vite:resolve',
      'vite:html-inline-proxy',
      'vite:css',
      'vite:esbuild',
      'vite:json',
      'vite:wasm-helper',
      'vite:worker',
      'vite:asset',
      'vite:wasm-fallback',
      'vite:define',
      'vite:css-post',
      'vite:build-html',
      'vite:worker-import-meta-url',
      'vite:asset-import-meta-url',
      'vite:force-systemjs-wrap-complete',
      'commonjs',
      'vite:data-uri',
      'vite:worker-post',
      'vite:dynamic-import-vars',
      'vite:import-glob',
      'vite:build-import-analysis',
      'vite:esbuild-transpile',
      'vite:terser',
      'vite:load-fallback'
    ],
    rollupOptions: {},
    getSortedPlugins: [Function: getSortedPlugins],
    getSortedPluginHooks: [Function: getSortedPluginHooks]
  },
  appType: 'spa',
  experimental: { importGlobRestoreExtension: false, hmrPartialAccept: false },
  getSortedPlugins: [Function: getSortedPlugins],
  getSortedPluginHooks: [Function: getSortedPluginHooks]
}
transforming...
2023-06-26T10:01:32.140Z vite:resolve 2.30ms src/styles/variables -> null
2023-06-26T10:01:32.179Z vite:resolve 8.30ms src/styles/mixins -> null
2023-06-26T10:01:35.065Z vite:resolve 5.75ms ../assets/fonts/Campton-Thin.woff -> /home/github/my_path_here/src/assets/fonts/Campton-Thin.woff
2023-06-26T10:01:35.066Z vite:resolve 6.06ms ../assets/fonts/Campton-ThinItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-ThinItalic.woff
2023-06-26T10:01:35.066Z vite:resolve 6.16ms ../assets/fonts/Campton-ExtraLight.woff -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraLight.woff
2023-06-26T10:01:35.066Z vite:resolve 6.21ms ../assets/fonts/Campton-ExtraLightItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraLightItalic.woff
2023-06-26T10:01:35.069Z vite:resolve 6.25ms ../assets/fonts/Campton-Light.woff -> /home/github/my_path_here/src/assets/fonts/Campton-Light.woff
2023-06-26T10:01:35.070Z vite:resolve 9.90ms ../assets/fonts/Campton-LightItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-LightItalic.woff
2023-06-26T10:01:35.070Z vite:resolve 10.43ms ../assets/fonts/Campton-Book.woff -> /home/github/my_path_here/src/assets/fonts/Campton-Book.woff
2023-06-26T10:01:35.071Z vite:resolve 10.87ms ../assets/fonts/Campton-BookItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-BookItalic.woff
2023-06-26T10:01:35.071Z vite:resolve 11.45ms ../assets/fonts/Campton-Medium.woff -> /home/github/my_path_here/src/assets/fonts/Campton-Medium.woff
2023-06-26T10:01:35.072Z vite:resolve 11.76ms ../assets/fonts/Campton-MediumItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-MediumItalic.woff
2023-06-26T10:01:35.072Z vite:resolve 11.87ms ../assets/fonts/Campton-SemiBold.woff -> /home/github/my_path_here/src/assets/fonts/Campton-SemiBold.woff
2023-06-26T10:01:35.072Z vite:resolve 12.17ms ../assets/fonts/Campton-SemiBoldItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-SemiBoldItalic.woff
2023-06-26T10:01:35.073Z vite:resolve 12.44ms ../assets/fonts/Campton-Bold.woff -> /home/github/my_path_here/src/assets/fonts/Campton-Bold.woff
2023-06-26T10:01:35.073Z vite:resolve 12.79ms ../assets/fonts/Campton-BoldItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-BoldItalic.woff
2023-06-26T10:01:35.074Z vite:resolve 13.04ms ../assets/fonts/Campton-ExtraBold.woff -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraBold.woff
2023-06-26T10:01:35.074Z vite:resolve 13.32ms ../assets/fonts/Campton-ExtraBoldItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraBoldItalic.woff
2023-06-26T10:01:35.074Z vite:resolve 13.65ms ../assets/fonts/Campton-Black.woff -> /home/github/my_path_here/src/assets/fonts/Campton-Black.woff
2023-06-26T10:01:35.075Z vite:resolve 14.00ms ../assets/fonts/Campton-BlackItalic.woff -> /home/github/my_path_here/src/assets/fonts/Campton-BlackItalic.woff
2023-06-26T10:01:35.374Z vite:resolve 0.35ms ../assets/fonts/Campton-ExtraLight.otf -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraLight.otf
2023-06-26T10:01:35.375Z vite:resolve 0.26ms ../assets/fonts/Campton-ExtraLightItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraLightItalic.otf
2023-06-26T10:01:35.376Z vite:resolve 0.26ms ../assets/fonts/Campton-Thin.otf -> /home/github/my_path_here/src/assets/fonts/Campton-Thin.otf
2023-06-26T10:01:35.377Z vite:resolve 0.31ms ../assets/fonts/Campton-ThinItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-ThinItalic.otf
2023-06-26T10:01:35.378Z vite:resolve 0.27ms ../assets/fonts/Campton-Light.otf -> /home/github/my_path_here/src/assets/fonts/Campton-Light.otf
2023-06-26T10:01:35.379Z vite:resolve 0.30ms ../assets/fonts/Campton-Book.otf -> /home/github/my_path_here/src/assets/fonts/Campton-Book.otf
2023-06-26T10:01:35.380Z vite:resolve 0.39ms ../assets/fonts/Campton-BookItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-BookItalic.otf
2023-06-26T10:01:35.381Z vite:resolve 0.26ms ../assets/fonts/Campton-LightItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-LightItalic.otf
2023-06-26T10:01:35.382Z vite:resolve 0.26ms ../assets/fonts/Campton-Medium.otf -> /home/github/my_path_here/src/assets/fonts/Campton-Medium.otf
2023-06-26T10:01:35.383Z vite:resolve 0.30ms ../assets/fonts/Campton-MediumItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-MediumItalic.otf
2023-06-26T10:01:35.384Z vite:resolve 0.21ms ../assets/fonts/Campton-SemiBold.otf -> /home/github/my_path_here/src/assets/fonts/Campton-SemiBold.otf
2023-06-26T10:01:35.385Z vite:resolve 0.27ms ../assets/fonts/Campton-SemiBoldItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-SemiBoldItalic.otf
2023-06-26T10:01:35.386Z vite:resolve 0.43ms ../assets/fonts/Campton-Bold.otf -> /home/github/my_path_here/src/assets/fonts/Campton-Bold.otf
2023-06-26T10:01:35.388Z vite:resolve 0.28ms ../assets/fonts/Campton-BoldItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-BoldItalic.otf
2023-06-26T10:01:35.393Z vite:resolve 0.55ms ../assets/fonts/Campton-ExtraBold.otf -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraBold.otf
2023-06-26T10:01:35.609Z vite:resolve 0.66ms ../assets/fonts/Campton-ExtraBoldItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-ExtraBoldItalic.otf
2023-06-26T10:01:35.610Z vite:resolve 0.34ms ../assets/fonts/Campton-BlackItalic.otf -> /home/github/my_path_here/src/assets/fonts/Campton-BlackItalic.otf
2023-06-26T10:01:35.611Z vite:resolve 0.41ms ../assets/fonts/Campton-Black.otf -> /home/github/my_path_here/src/assets/fonts/Campton-Black.otf
2023-06-26T10:01:41.105Z vite:resolve 1.19ms src/styles/_variables.scss -> null
2023-06-26T10:01:41.134Z vite:resolve 18.07ms src/styles/_mixins.scss -> null
✓ 13702 modules transformed.

Validations

@dd-jonas
Copy link
Contributor Author

I forced rollup@3.10.0 using yarn's selective dependency resolution but that didn't fix it either.

@patak-dev Do you have more experience with this?

@dd-jonas
Copy link
Contributor Author

I investigated further and checked the heap allocation of vite build using Clinic Doctor. Looks like a terrible performance issue since 4.2.0, the heap used simply doubled.

Vite 4.1.5:
vite@4 1 5

Vite 4.3.9 (tested with 4.2.0 as well, graph looks similar)
vite@4 3 9

As mentioned before I forced yarn to use rollup 3.10.0, which was the version Vite used before 4.2.0, but I don't see any difference. I'm assuming there's a significant performance issue introduced in Vite 4.2.0.

@bluwy bluwy added p4-important Violate documented behavior or significantly improves performance (priority) performance Performance related enhancement and removed pending triage labels Jun 27, 2023
@dd-jonas
Copy link
Contributor Author

dd-jonas commented Jun 28, 2023

Update: Just to be clear, the out-of-memory issue is because my CI environment does not have much memory. The question is why Vite 4.2.0 uses much more memory than before.

Did some more tests and ran the heap profiler with and without --sourcemap false with version 4.1.5 and 4.3.9.

--sourcemap Vite 4.1.5 Vite 4.3.9 
true 3,300 MB 3,300 MB
false 1,750 MB 3,300 MB

I'm not sure why, but I'm getting different results when building without sourcemaps on 4.3.9. Either the graph looks exactly as if I'm building with sourcemaps (3,300 MB), but sometimes it lowers to 2,500 MB. (Edit: I've run the build command many times now, and now it's always 3,300 MB like the screenshot above. Maybe some unexpected optimization happened that caused it to drop for a while?)

Either way, I guess something has changed related to sourcemaps, as the results with are nearly identical.

@dd-jonas
Copy link
Contributor Author

@bluwy Found the issue finally. This commit broke the CLI option --sourcemap false, as it converts false to a string. This can be checked when running vite build --debug --sourcemap false, you will see it still outputs sourcemaps.

@bluwy
Copy link
Member

bluwy commented Jun 28, 2023

Thanks for figuring this out @dd-jonas (and sorry for the breakage). Didn't expect this to happen and feels like a weird behaviour from cac, it wants us to support --no-sourcemap instead, but I don't think that's ideal.

Would you like to send a PR fixing this? The changes should be here:

function cleanOptions<Options extends GlobalCLIOptions>(
options: Options,
): Omit<Options, keyof GlobalCLIOptions> {
const ret = { ...options }
delete ret['--']
delete ret.c
delete ret.config
delete ret.base
delete ret.l
delete ret.logLevel
delete ret.clearScreen
delete ret.d
delete ret.debug
delete ret.f
delete ret.filter
delete ret.m
delete ret.mode
return ret
}

Handling the "false" as false instead. Or I can tackle this tomorrow too.


You can also workaround this for now but setting the config at build.sourcemap: false instead in vite.config.js. (and remove the CLI arg)

@dd-jonas
Copy link
Contributor Author

@bluwy Cheers, I opened a PR here #13663. Feel free to suggest a different approach if you think there's a better way to tackle this.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 14, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
p4-important Violate documented behavior or significantly improves performance (priority) performance Performance related enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants