Webpack module federation breaks with multiple entrypoints

[codepunkt]

This time it's @sokra who told me this is probably the place to open an issue.

• Operating System: Windows 10
• Node Version: 12.16.2
• NPM Version: 6.14.4
• webpack Version: 5.0.0-beta.22
• webpack-dev-server Version: 3.11.0
• Browser: Chrome 84.0.4147.105, Firefox 79.0, Edge 84.0.522.48

• This is a bug

Code

https://github.com/codepunkt/module-federation-examples/tree/dynamic-host-remote

Expected Behavior

Using webpack-dev-server instead of webpack should still support module federation with additional content tacked onto the remoteEntry by defining it as an additional entry.

Actual Behavior

Running "yarn start" to start webpack-dev-server breaks module federation and thus breaks the app in development mode.

For Bugs; How can we reproduce the behavior?

1. clone repository
2. ensure you're on branch "dynamic-host-remote"
3. run yarn on repo root
4. go into "dynamic-host-remote" directory
5. run yarn start in "dynamic-host-remote" directory
6. open localhost:3001 in the browser
7. encounter an error in the browser console that happens when executing app2/remoteEntry.js with the additional contents that were added to this entrypoint by webpack-dev-server
8. OPTIONAL: run yarn build && yarn serve and revisit localhost:3001 to see production build working just fine.

[alexander-akait]

@webpack-bot move to webpack/webpack-dev-server My mistake

[alexander-akait]

Feel free to send a fix

[codepunkt]

If I had a clue what the problem is, i'd send a PR right away. Spent a few hours with this, don't have the slightest idea 🤔

[alexander-akait]

/cc @sokra Any ideas on this?

[ScriptedAlchemy]

Okay, so the issue here is that WDS is likely appending its own entrypoint to the array / injecting a "require" to the hmr client. That typically serves as the entry module of an application. When ModuleFederation emits a remoteEnty with the same name as the entrypoint, they are merged.

This lets us attach initialization code, to federated remotes. something like changing publicPath.

webpack_public_path = new URL(document.currentScript.src).origin + "/";

module-federation/module-federation-examples#277

The remote becomes 400kb big (usually like 5kb) and the module returned to the global is not __webpack_require__("webpack/container/entry/app1"); - therefore window.app1 will not be __webpack_require__("webpack/container/entry/app1"); and the federated API is never set to the global

@sokra i believe that either the WDS entry needs to return the container - which may lead to HMR capabilities? or the WDS entry module should not be applied to federated entry points when combined.

module.exports = {
  entry: {
    app1: "./src/setPublicPath",
    main: "./src/index",
  },
plugins: [
    new ModuleFederationPlugin({
      name: "app1",
      remotes: {
        app2: "app2@http://localhost:3002/remoteEntry.js",
      },
      shared: { react: { singleton: true }, "react-dom": { singleton: true } },
    }),
    new HtmlWebpackPlugin({
      template: "./public/index.html",
    }),
  ],

[sokra]

The wds entrypoint should be prepended instead of appended. The last value is exported.

[ScriptedAlchemy]

Will take a look, thanks @sokra

[ScriptedAlchemy]

Looks like we want to modify this

 const prependEntry = (originalEntry, additionalEntries) => {
      if (typeof originalEntry === 'function') {
        return () =>
          Promise.resolve(originalEntry()).then((entry) =>
            prependEntry(entry, additionalEntries)
          );
      }

      if (typeof originalEntry === 'object' && !Array.isArray(originalEntry)) {
        /** @type {Object<string,string>} */
        const clone = {};

        Object.keys(originalEntry).forEach((key) => {
          // entry[key] should be a string here
          const entryDescription = originalEntry[key];
          if (typeof entryDescription === 'object' && entryDescription.import) {
            clone[key] = Object.assign({}, entryDescription, {
              import: prependEntry(entryDescription.import, additionalEntries),
            });
          } else {
            clone[key] = prependEntry(entryDescription, additionalEntries);
          }
        });

[sokra]

It already prepends... So there is somethings else wrong

[sokra]

The bug is here:

webpack-dev-server/lib/utils/addEntries.js

Lines 86 to 92 in 4ab1f21

	/** @type {Entry} */
	const entriesClone = additionalEntries.slice(0);
	[].concat(originalEntry).forEach((newEntry) => {
	if (!entriesClone.includes(newEntry)) {
	entriesClone.push(newEntry);
	}
	});

Desprite the function name prepend it actually appends...

Change it to:

      /** @type {Entry} */
      const entriesClone = [].concat(originalEntry);
      additionalEntries.forEach((newEntry) => {
        if (!entriesClone.includes(newEntry)) {
          entriesClone.push(newEntry);
        }
      });

Could somebody send a PR for that? As test case you can use output.library = "TEST" and check if this created the correct global.

[alexander-akait]

Great, feel free to send a PR

[snitin315]

I will send a PR.

[codepunkt]

I'm not sure this will actually fix HMR with module federation - but i'm looking forward to try!

[ScriptedAlchemy]

Opening PR, will need to work on / help on test case

[knagaitsev]

I looked into this more closely, and here is the full problem:

In most cases, the dev server is trying to add some variation of these 2 entries (unless you disable their injection): webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js.

Current behavior (webpack 5):

Our config has a single entry: main.js.

The 2 dev server entries are appended to the entry list (as explained above), resulting in something like:

main: {
  import: {
    'main.js',
    'webpack-dev-server/client/default/index.js',
    'webpack/hot/dev-server.js'
  }
}

Then we call the entryOption hook with all of these entries here:

webpack-dev-server/lib/utils/updateCompiler.js

Line 51 in 4ab1f21

compiler.hooks.entryOption.call(config.context, config.entry);

This results in an entry list on the webpack side that looks like this:
main.js, main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js

So it would seem that prepending the 2 entries would fix the issue:
main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js, main.js

However, it is not that simple because webpack essentially filters out the duplicate entries, taking only the first of the duplicates (https://github.com/webpack/webpack/blob/bdf4a2b942bed9d78815af828f7935ddfcd3d567/lib/Compilation.js#L1764), so we get: main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js (regardless of if we append or prepend in the webpack step). The very last in this list is currently the one that is exported, so in no case is it ever main.js while there are other entries being injected.

Thus, the solution is to either change how this filtering works, or change the selection of entries by webpack such that the very first entry from the entryOption hook is used, rather than the very last.

I think the latter is preferred, because I realize now that it is not ideal to be pushing duplicate entries into the entryOption hook (as we do currently). Instead, we should only be pushing webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js into the entryOption hook, as they have not yet been registered as entries. On the webpack side, these entries should not be considered as the module exports, as they were added later after the initial configuration entries.

[sokra]

Then we call the entryOption hook with all of these entries here:

webpack-dev-server/lib/utils/updateCompiler.js

Line 51 in 4ab1f21

compiler.hooks.entryOption.call(config.context, config.entry);

Why does webpack-dev-server do that? The hooks is not owned by webpack-dev-server. It should not call it. It's already called by webpack itself. That's probably also the reason why the entries are added twice...

Seems like this seem to be some kind of hack to reapply the entry option, because it has been modified after the options has been applied, which is too late. It doesn't make sense to modify the options after they have been applied to the compiler (converted to plugin). You need to use a plugin instead.

A plugin to add these entries for webpack-dev-server could be like that:

compiler.hooks.make.tapAsync({
    name: "webpack-dev-server",
    stage: -100
}, (compilation, callback) => {
    const options = {
        name: undefined // global entry, added before all entries
    };
    const dep = EntryPlugin.createDependency("webpack-dev-server/client/...", options);
    compilation.addEntry(context, dep, options, err => {
        callback(err);
    });
});

Note the hack works in webpack 4 as each entrypoint can only have a single module (arrays are wrapped in a artificial module), so reapplying the entryOption also adds it twice, but the second one overrides the first one.

There is no global entry in webpack 4, so the above plugin doesn't work for webpack 4. I would keep the hack as legacy code, as a clean solution is more complex.

[codepunkt]

Thanks everyone for chiming in.

What confuses me a little about this is that webpack-dev-server, which i always thought of as a "major part of webpack", actually seems to be an afterthought when it comes to new developments.

I don't understand a lot of the details everyone wrote in this issue, but i'm glad this problem is being ironed out and fixed.

[tianyingchun]

could you give suggestions?

[alexander-akait]

No idea, maybe non official plugin break, hash should not changed on initial run, please create reproducible test repo

[tianyingchun]

ok i have created demo repo for reproduce this issue.

[alexander-akait]

Great, can you provide a link?

[tianyingchun]

wait me moment, it seem that i can't commit github now, authentication failed.
https://github.com/tianyingchun/webpack-dev-server-demo-2692

[tianyingchun]

i have updated repo demo here please check it .
https://github.com/tianyingchun/webpack-dev-server-demo-2692

1. npm install
2. node serve.js
3. open url: http://localhost:5000

you can see the page infinite loop reload now

[alexander-akait]

Thanks, I will look at this in near time

[alexander-akait]

oh, you have multiple hmr with different hashes, it is weird usage, to be honestly, ideally you should run dev server on each compiler

[alexander-akait]

we plan for webpack-dev-server@4.1 implement plugin support so you will put plugins: [new DevServer(options)] for the each compiler

[alexander-akait]

Or use multiple entries...

[alexander-akait]

i.e. you have only one hmr code and web socket connection, but handle them different compilers (so web socket gets hashes from first and from second compilers), it is not safe generally, before it works only because we have bug in runtime logic, after fix it you got the problem

[alexander-akait]

Can you clarify why do you use single dev server for multi compiler mode? What is use case? Why do not run two dev servers?

[tianyingchun]

yes, we have huge complex project with multiple modules in one repo, we setup configurations to allow us start serve for these individual modules one or more ,because these modules is individual SPA application;

1. we want to startup one server with the same host && port like http://localhost:5000/
2. we attached customized middleware in dev Server pipeline /pages/*
3. we don't want to start serve for each module with different dev server, because it has different access url.
4. also has more reason, because webpack it has support multi compilers, i think dev-server should support we pass multi compilers with hot

[tianyingchun]

we plan for webpack-dev-server@4.1 implement plugin support so you will put plugins: [new DevServer(options)] for the each compiler

for this, do dev-server start only one or multi dev server for multi modules?

[alexander-akait]

also has more reason, because webpack it has support multi compilers, i think dev-server should support we pass multi compilers with hot

Sorry, it is impossible by design, you can't have the same HMR code for multi compilers, it is invalid and weird, before you don't have bug due our invalid logic for initial, in fact you should have gotten this problem much earlier, we support multi compiler mode, but your case has multiple independent builds with the same HMR, even more new features like lazyCompilation, buildless mode (future) will not work too

[alexander-akait]

Even more, you have only one web socket connection, therefore hash from one build will be already rewrite hash from other build, in theory we can pass unique name of compiler and implement multiple statuses, but it is more work then you think, also many plugins (for dev server) can break your HMR

[alexander-akait]

yes, we have huge complex project with multiple modules in one repo, we setup configurations to allow us start serve for these individual modules one or more ,because these modules is individual SPA application;

in this case you should use multi entry mode (it will optimize your code even better) or module federation

[tianyingchun]

ok, i have rewrite my codebase to startup multi dev-server with multi port and dynamic organize them into express server page proxy.

BTW it seems that we can't disabled logger.info() message via dev-server config, or webpack stats.logging= false.

[tianyingchun]

And does webpack-dev-server provider @types for v4.0.0?

[snitin315]

And does webpack-dev-server provider @types for v4.0.0?

No, but it is in our roadmap.

[alexander-akait]

@tianyingchun for logger we have the special option https://webpack.js.org/configuration/other-options/#level, we decide don't have a lot of different logger options as were before and focus only on one

[IgorStetsiuk]

This time I'll be that guy... do we have any updates on this?

Have one question regarding Hot Module Replacement. Seems it doesn’t work properly. I used CRA with react-scripts v 5.0.0 and CRACO to override the webpack config in order to set up my module federation and that’s everything is fine (build, test, start commands)… Thought when it comes to the local development I need manually hard reload pages to reflect changes, and it’s really inconvenient.

I'm thinking to build React app from a scratch and playing around with webpack config but that is not what I wanted to do as we already have a pretty well config provided by CRA only need to override it for Module Federation config is that correct?

Does somebody know whether CRA (latest version 5) is able to work with hot module reload? What configuration do I miss?
Thank you!

[ScriptedAlchemy]

This time I'll be that guy... do we have any updates on this?

Have one question regarding Hot Module Replacement. Seems it doesn’t work properly. I used CRA with react-scripts v 5.0.0 and CRACO to override the webpack config in order to set up my module federation and that’s everything is fine (build, test, start commands)… Thought when it comes to the local development I need manually hard reload pages to reflect changes, and it’s really inconvenient.

I'm thinking to build React app from a scratch and playing around with webpack config but that is not what I wanted to do as we already have a pretty well config provided by CRA only need to override it for Module Federation config is that correct?

Does somebody know whether CRA (latest version 5) is able to work with hot module reload? What configuration do I miss?

Thank you!

This is a very hard problem to solve. An easy workaround I use is I'll fetch all remote urls on a poll and hash them. If any hash based on the contents changes, I window reload the page. Simple but easy way to get live reloading.

HMR is not easy since webpack needs to know both where something exists and where it's consumed. All graphs would need to be interlinked

[IgorStetsiuk]

@ScriptedAlchemy Hello! Thank you for your reply. Wouldn't you mind sharing a piece of that config where you "fetch all remote URLs on a poll and hash them"? I'm not super aware of how to set up such config in webpack and it sounds a bit difficult to me to implement that so maybe some samples would be really helpful to me to understand that configuration and make live reloading in my case. Thank you again!

[ScriptedAlchemy]

Fetch(jsUrl).then(text).then(md5hashFronString)

Store it on some object and put the fetch in a setInterval.

[xujie-phper]

any updates, i face the same issue when use module federation

[rdenman]

Since this issue is still open, I'll link to my (pretty damn hacky) "solution" for HMR I posted in another thread: webpack/webpack#11240 (comment)

It's very far from perfect, but it does the job in a pinch

[ssuvorov]

20 October 2023 👀