feat: building without node_modules

[aleksandrjet]

Changes

I added the isIndependent: true option to the NodeJS adapter. With this option, when building, the output folder dist/server will contain all the necessary data to start the server. And your server will be able to work without the node_modules folder.

For what

Including external dependencies in the build helps reduce the size of the application. In new example, the weight of the docker image with the application with the node modules folder is 244 MB, and without it - 135 MB.

At the same time, if you remove size of the os and NodeJS from the comparison, then the difference is much more significant - 110 MB versus 500 KB!

You can see more details about the difference in docker images in the new example.

All this can be configured in astro.config.mjs yourself. Why make a new parameter?

• This will help lower the entry barrier. The functionality requires some skill in working with Vite and NodeJS, since you cannot simply pass external: false to the config and get result
• Without adding the parameter, users will have to add a separate configs for the build and run dev-server
• NextJS has a similar parameter and users will be comfortable switching from NextJS to Astro. This option is used by those who do not want to deploy to Vercel. I think these are the people who are potential future Astro users.

Testing

I added tests to the integrations/node folder. These tests checks functionality after deleting node_modules folder

Docs

• If this is accepted, then I think we will need to update the docker image deployment documentation, as well as update the build documentation.
• Perhaps the parameter name should be changed

cc @sarah11918

[changeset-bot]

🦋 Changeset detected

Latest commit: 83a7ca2

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[lilnasy]

This would be a great addition, thanks for taking this on!

• Would this work with react and lit integrations? Both of them add their libraries' SSR code as external.
• Would errors about node:fs being imported into client code be silenced? rollupOptions act on both server and client bundles, and we are allowing node modules for both of them.
• Would this work with markdown-remark/prismjs ecosystem, which is astro code heavily uses? There are frequent uses of CommonJS features that won't work once compiled into modules.

[aleksandrjet]

Would this work with react and lit integrations? Both of them add their libraries' SSR code as external.

Yes, in the current implementation I added isIndependent option to NodeJS adapter and if some framework adds its own external configuration, then it is combined with the previous one. In the case of react exteranl, the field will look something like this: ['react', 'react-dom', 'node:fs', '...some node modules'].

To avoid this, I plan to move the option from the NodeJS adapter to the asto config, to experimental.isIndependent. This option will force only NodeJS modules to be left external. I need a little time to do this.

If anyone sees weaknesses in this implementation, welcome to the discussions, I will be glad to receive any feedback :)

[bluwy]

For the current PR's approach, I sent an upstream PR (vitejs/vite#16019), that should simplify the configuration, so simply setting vite.ssr.noExternal: true should be enough. And I don't think we need to introduce a new option, but maybe docs is enough.

To avoid this, I plan to move the option from the NodeJS adapter to the asto config, to experimental.isIndependent.

This is true though. My PR won't solve this, and the workaround would be to force ssr.external to only be the node builtins with a Vite plugin configResolved hook. However, I don't think it's safe to do this only for node builtins. For example, we cannot bundle sharp because it contains binaries, so that needs to be excluded. Users could also use any binaries that can't be bundled, so it will be problematic.

Instead I think we should check each ssr.external configuration in this repo and remove the unnecessary ones. From a quick glances, I'm not sure why some deps were forced externalized. Or perhaps they only need to be externalized in dev.

[ematipico]

Thank you, @aleksandrjet, for this great addition. This PR requires updating the documentation too: https://docs.astro.build/en/guides/integrations-guide/node/

Will you do that too? Let us know if you have the capacity. If not, we will do that

[ematipico]

I would call the option independent. Also, it requires some documentation

[ematipico]

Suggested change

	Added the `isIndependent: true` option to the `NodeJS` adapter. With this option, when building, the output folder `dist/server` will contain all the necessary data to start the server. And your server will be able to work without the `node_modules` folder
	Added the `isIndependent: true` option. With this option, when building, the output folder `dist/server` will contain all the necessary data to start the server. And your server will be able to work without the `node_modules` folder

The package @astrojs/node is already the adapter, I think it's redundant to say "to the Node.js adapter".

[aleksandrjet]

I plan to move this to experimental config:

export default defineConfig({
	output: 'server',
	adapter: node({
		mode: 'standalone',
	}),
	experimental: {
		isIndependent: true,
	}
});

[ematipico]

In this PR?

[aleksandrjet]

Yes, I have now moved this option to config.experimental.isIndependent. Without this change, building without node_modules will not work for some frameworks

[aleksandrjet]

Will you do that too? Let us know if you have the capacity. If not, we will do that

I will be glad if you help with docs, but previously we need to wait PR of @bluwy

[bluwy]

I was thinking that we shouldn't need this PR or option at all. Once vitejs/vite#16019 lands, we only need to document that they can set vite.ssr.noExternal: true if they want to bundle everything.

However of course, there are integrations today that force set vite.ssr.external to something. I don't think they are needed so we should probably follow-up instead on these usages and remove them if it makes sense. For example, as explained above, some packages like sharp can't be bundled at all so they should always be set in vite.ssr.external. That we could document that it needs to be installed if you were to bundle everything.

[matthewp]

Thanks for the contribution, but after discussing it I think we're going to wait on the Vite option and avoid making the Node adapter more complicated.