Why does the preserveModules option generate node_modules folder

[ghbakhtiari]

Documentation Is:

• Missing
• Needed
• Confusing
• Not Sure?

As described here in the docs, I'm using the plugin-node-resolve plugin to include the dependencies of my library in the final bundle.

I'm also using the external option as described here to exclude the peerDependencies. So the final rollup config looks something like this:

import path from 'path';
import resolve from '@rollup/plugin-node-resolve';
import babel from '@rollup/plugin-babel';

export default {
  input: 'src/index.js',
  output: {
    dir: './dist',
    format: 'es',
    preserveModules: true,
  },
  external: [
    'prop-types',
    'react',
    'react-dom',
  ],
  plugins: [
    resolve({
      extensions: ['.js', '.jsx', '.json'],
    }),
    babel({
      babelHelpers: 'bundled',
      exclude: 'node_modules/**',
    }),
  ],
};

The problem is that after the build, the dependencies of the package are generated in this location: dist/node_modules/PACKAGE_NAME. And their import statements in other files are something like this:
import SOMETHING from '../../../node_modules/PACKAGE_NAME/index.js';

And on the other hand, when I build the package on the consumer project (using prepare script), this nested node_modules directory doesn't exist (possibly because npm and yarn flatten the node_modules structure).

So I was wondering is there any option to fix this problem? e.g. a way to change the name of these generated node_modules folders and their import statements to something else...

[lukastaegert]

The folder is created because preserveModules is preserving the original file names and directory structure. It does not understand that node_modules is something special, though. Also, all exports are rewritten in a way that does not depend on Node's dependency resolution algorithm.

If you want to change names, you might have some luck using the entryFileNames option with a function. I am not 100% sure we implemented it for preserveModules, would be interesting if you gave it a go: https://rollupjs.org/guide/en/#outputentryfilenames

[ghbakhtiari]

@lukastaegert Thanks for the reply.

I just tried entryFileNames both with and without the preserveModules option, and unfortunately got this error (both times):

[!] TypeError: name.replace is not a function

In the error's stack trace it seems that the entryFileNames is passed as pattern and finally produces an error in this function:

export function sanitizeFileName(name: string): string {
  return name.replace(/[\0?*]/g, '_');
}

It's a bit strange because the docs just says that this option supports both string and function 🤔 Am I doing something wrong?!

[lukastaegert]

Did the function return a string? 😉

[lukastaegert]

And are you on latest Rollup?

[ghbakhtiari]

Yes it's returning some random string 😀 The important problem is that I put some logs inside the function and it's not called at all.

The versions are:

    "@rollup/plugin-alias": "^3.1.1",
    "@rollup/plugin-babel": "^5.0.4",
    "@rollup/plugin-commonjs": "^13.0.0",
    "@rollup/plugin-node-resolve": "^8.1.0",
    "@rollup/plugin-replace": "^2.3.3",
    "rollup": "^2.18.1",
    "rollup-plugin-terser": "^6.1.0"

[lukastaegert]

As a first step I would update Rollup as that feature is a little newer than your minimum Rollup version: https://github.com/rollup/rollup/releases/tag/v2.20.0

[ghbakhtiari]

The update fixed the function problem 👍

But now when I try to rebuild the full path (to replace the node_modules with something else) with something like this:

    entryFileNames: ({ facadeModuleId }) => {
      if (!facadeModuleId.includes('node_modules')) {
        return '[name].js';
      }

      return facadeModuleId.replace('node_modules', 'dependencies');
    },

I get this error:

[!] Error: Invalid pattern "/SOME_FOLDERS/dependencies/SOME_PACKAGE/index.js" for "output.entryFileNames",
patterns can be neither absolute nor relative paths and must not contain invalid characters.

It seems that I can only nest the result in some sub_directories but can't change the path that the file already has.

For example if I write it like this:

    entryFileNames: ({ facadeModuleId }) => {
      if (!facadeModuleId.includes('node_modules')) {
        return '[name].js';
      }

      return 'dependencies/[name].js';
    },

then it will nest all of the node_modules/SOME_PACKAGE/... files in node_modules/SOME_PACKAGE/dependencies/....

[lukastaegert]

Ok, this should probably be improved. I would have thought that [name] contains the full path, but apparently it is prepended. Not ideal. For now, you can manually convert the facadeModuleId to a valid path fragment that does not start with a . or /, i.e. /some/path/node_modules/lib/file.js -> dependencies/lib/file.js. Node's path.relative can be helpful here.

[lukastaegert]

Ah, that will still not solve your problem, my bad

[ghbakhtiari]

Exactly, the problem is that whatever we return from this function, it would only be appended to the end of the path of the file.

[lukastaegert]

I fear we cannot fix this without a breaking change at the moment

[ghbakhtiari]

Well, currently the isPlainPathFragment function returns false when the name starts with "/", "./" or "../", and then throws an error in renderNamePattern.

So instead of throwing, if we just add this functionality where these types of names would change the path as well, would it still be a breaking change?
I mean in absolute cases (starting with "/"), the path would be completely replaced. And in the relative cases (starting with "./" or "../") a new path would be resolved from them...

Also, another way is to add a new option besides entryFileNames which supports this functionality (change the path as well...)

[davidroeca]

I'm encountering this issue, although it's slightly different in that I'd also like to have control over the modules that get output from preserveModules: true. Right now, my src/ directory path is preserved, so I end up with dist/src/index.js when I really only want dist/index.js.

I think the underlying issue here is with @rollup/plugin-node-resolve. See rollup/plugins#321 and rollup/plugins#277, both of which have been closed due to inactivity.

I can try reproducing soon, but right now my work-around has been to use rollup-plugin-rename to restructure the output, and also to handle the relative imports that otherwise would have problems due to this restructuring.

[Lazyuki]

I was having the same issue and since I didn't know if it was a rollup bug or not I just created a plugin to fix this called rollup-plugin-rename-node-modules. I'm using it in my project and now it works.

[shrpne]

I'm encountered same issue.

preserveModules: false gives this

dist/
├── index.js
├── second.js

preserveModules: true gives this

dist/
├── src/
│   ├── index.js
│   ├── second.js
├── node_modules/

To workaround it, I combined davidroeca's rollup-plugin-rename and Lazyuki's node_module renaming to external. The final solution:

import commonjs from '@rollup/plugin-commonjs';
import resolve from '@rollup/plugin-node-resolve';
import babel from 'rollup-plugin-babel';
import rename from 'rollup-plugin-rename';

export default {
    input: 'src/index.js',
    output: {
        dir: 'dist/',
        format: 'cjs',
        preserveModules: true,
    },
    plugins: [
        commonjs(),
        resolve({
            resolveOnly: ['lodash-es'],
            // modulesOnly: true,
        }),
        babel({
            runtimeHelpers: true,
        }),
        rename({
            include: ['**/*.js'],
            map: (name) => name
                .replace('src/', '')
                .replace('node_modules/', 'external/')
                .replace('../../external', '../external'),
        })
    ],
};

And the output will look like this:

dist/
├── index.js
├── second.js
├── external/

[Quirksmode]

@shrpne Thank you so much, this was causing me so much pain trying to solve!!

It's really worrying that this is the fix, assuming this is just a temporary workaround as this must be a bug that needs fixing?

For me the two issues are that it creates an src folder and after fixing that, it's unable to properly find the node_modules folder

[tbaustin]

Any progress on this?

[kirkobyte]

I'm experiencing this issue also. Adding this to my config has worked pretty well for me in the meantime:

const pkg = require('./package.json');

const externalPackages = [
  ...Object.keys(pkg.dependencies || {}),
  ...Object.keys(pkg.peerDependencies || {}),
];
// Creating regexes of the packages to make sure subpaths of the
// packages are also treated as external
const regexesOfPackages = externalPackages
  .map(packageName => new RegExp(`^${packageName}(\/.*)?`));

export default {
  external: regexesOfPackages,
  // ...
};

[wangjia184]

I am using the following code to solve my issue

function entryFileNames(chunkInfo) {

	var matches = /(?:\bsrc[\\|\/])(?<path>.*?[^\\\/]+)(?:\.svelte)$/g.exec(chunkInfo.facadeModuleId);
	if( matches != null && matches.length > 1 ){
		return matches[1] + ".js";
	}

	return "[name].js";
}

{
		input: ['src/Widget1.svelte', 'src/Widget2.svelte', ...],
		output: {
			sourcemap: true,
			format: 'amd',
			name: 'app',
			dir: 'public/build/',
			entryFileNames: entryFileNames
		}
}

[lukastaegert]

Fix at #4565

[lukastaegert]

This issue has been resolved via #4565 as part of rollup@3.0.0-3. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-3 or npm install rollup@beta. It will likely become part of a regular release later.

[lukastaegert]

This issue has been resolved via #4565 as part of rollup@3.0.0-4. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-4 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4565 as part of rollup@3.0.0-5. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-5 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4565 as part of rollup@3.0.0-6. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-6 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4565 as part of rollup@3.0.0-7. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-7 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4565 as part of rollup@3.0.0-8. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-8 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4565 as part of rollup@3.0.0. You can test it via npm install rollup.

[RedTn]

Fix at #4565

@lukastaegert Hi, I tried updating to rollup v3, but we are unclear what exactly was resolved? From my testing it still outputs a node_modules folder, so is the fix something else?

[lukastaegert]

Yes, but you can change the path now by supplying a function for entryFileNames. Citing the original issue

a way to change the name of these generated node_modules folders and their import statements to something else...

is now available. Depending on your actual problem, output.preserveModulesRoot might also be useful.

[RedTn]

Gotcha, entryFileNames works for me, thanks

[oztek22]

for those who are still struggling with node_modules issue, this output object should fix it. thanks @lukastaegert for fixing it ✌🏻.

{
      dir: 'lib',
      format: 'esm',
      sourcemap: true,
      preserveModules: true,
      preserveModulesRoot: 'src',
      entryFileNames: (chunkInfo) => {
        if (chunkInfo.name.includes('node_modules')) {
          return chunkInfo.name.replace('node_modules', 'external') + '.js';
        }

        return '[name].js';
      }
    }

[eightHundreds]

chunkInfo.name.replace('node_modules', 'external') + '.js';

// pnpm projects will have multiple node_modules folders.
chunkInfo.name.replace(/node_modules/g, 'external') + '.js';