Websocket server disconnects clients too early causing reload loop

[koggdal]

• This is a bug
• This is a modification request

Code

No code available for this issue, see description further down for details.

Please paste the results of npx webpack-cli info here, and mention other relevant information

System:
OS: macOS 11.4
CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Memory: 22.59 GB / 64.00 GB
Binaries:
Node: 14.17.5 - ~/.nvm/versions/node/v14.17.5/bin/node
Yarn: 1.19.1 - ~/.yvm/.yarn/shim/yarn
npm: 6.14.14 - ~/.nvm/versions/node/v14.17.5/bin/npm
Browsers:
Chrome: 93.0.4577.63
Firefox: 91.0.2
Safari: 14.1.1
Monorepos:
Yarn Workspaces: 1.19.1
Packages:
compression-webpack-plugin: 8.0.1 => 8.0.1
css-minimizer-webpack-plugin: 3.0.2 => 3.0.2
html-webpack-harddisk-plugin: ^2.0.0 => 2.0.0
html-webpack-plugin: 5.3.2 => 5.3.2
terser-webpack-plugin: 5.2.3 => 5.2.3
webpack: 5.52.0 => 5.52.0
webpack-bundle-analyzer: 4.4.2 => 4.4.2
webpack-cli: 4.7.2 => 4.7.2
webpack-dev-middleware: 5.0.0 => 5.0.0
webpack-dev-server: 4.2.0 => 4.2.0
workbox-webpack-plugin: 6.2.4 => 6.2.4

Expected Behavior

Page load once and doesn't reload (once) until a change is made.

Actual Behavior

Page reloads infinitely in an endless loop. You open the site in the browser and the page starts reloading in a loop immediately when loaded.

For Bugs; How can we reproduce the behavior?

I believe this problem only occurs in large projects. We have a big monorepo with multiple products all running on the same domain and built in the same Webpack step.

I have built webpack-dev-server locally and narrowed it down to this single commit that introduced the issue: a6501e6

When using the previous commit of webpack-dev-server in our project, the issue is not there. With this commit, the issue is there and we get a reload loop.

I believe this is because the commit changed heartbeatInterval from 30 seconds to 1 second, and if the client isn't alive it terminates the client. In the browser logs for the reloads we can see a "Disconnected!" message and then it reloads. In our case, I'm thinking that sometimes things are too slow to start up within one second, and webpack-dev-server then disconnects, reloads, waits 1 second, disconnects... in a loop.

I have tried modifying this timer locally, and both 20 seconds and 5 seconds were fine for us, and didn't cause any reload loop. I'm not sure of the reasoning for lower this heartbeat from 30 seconds to 1 second, but if it's okay to keep it high, I think high would make sense to not cause instability for others depending on computer speed etc.

Let me know if you need any more details! Thank you.

For Features; What is the motivation and/or use-case for the feature?

[alexander-akait]

Duplicate #3803

[koggdal]

@alexander-akait Is it a true duplicate? I see no mention of these details in the other one. Nothing about that commit or the heartbeat timer for the Websocket server?

[alexander-akait]

If you don't think it is the same please provide example, please avoid pointing to code, in your link nothing was changed, it is just internal refactor, problems like infinity loop/doesn't work for me/stack overflow/etc is not help and very hard to debug

[alexander-akait]

Raising the timer value never solves the problem, since the delays on your side may increase in the future and the problem will reappear

[koggdal]

@alexander-akait In the description above I mention that the heartbeat timeout has been changed from 30 seconds to 1 second, which is too short in big projects. This detail is not an internal refactor, but a big change that causes the reload problems for us. I've analyzed this in a detailed manner locally, reproducing it with this commit and not the previous commit. I reproduce it with static heartbeatInterval = 1000; but not with static heartbeatInterval = 30000;.

I recognize that this is a timing issue, but is it a working assumption that the page will always on all computers load within 1 second? Having a higher timeout would make this much more stable either way.

[koggdal]

To dive into the timing change more, what is the reason for picking 1 second instead of 30 seconds? What would be the downside from having a higher value?

[alexander-akait]

higher value is not fix the problem, delay can be more on your side, there is no point in reasoning about value...

[alexander-akait]

Something wrong in state maybe, and we should improve checks, like no try to reconnect, if web socket is not connected or something else

[koggdal]

@alexander-akait Is there a common case where pages take 30 seconds to load, compared to more than 1 second? If there is little downside to putting a higher value, and less users of Webpack will have reload issues, I'm thinking it may be a worthwhile tradeoff?

If the timeout can't be changed, what do you suggest as another solution for the problem? The problem is the timing specifically. The page takes some time to load up all the things, so I'm not sure what else would be a working solution.

[alexander-akait]

If the timeout can't be changed, what do you suggest as another solution for the problem? The problem is the timing specifically. The page takes some time to load up all the things, so I'm not sure what else would be a working solution.

as you can see we have isUnloading https://github.com/webpack/webpack-dev-server/blob/master/client-src/index.js#L14, so logic for reconnect should be combined with this, again tomorrow you will have 1 minutes to load (yes it is exotic), but it is posibble, yes we can add new option for client if you need increase this, but I still thinks it is only workaround

[koggdal]

A new option would be great, if that allows us to keep using Webpack! 👍 🙏

If you see this as a workaround, what would a proper solution be?

[alexander-akait]

If you see this as a workaround, what would a proper solution be?

Need fix our runtime logic, I describe it above, why dev start starts slow, do you have something non standard?

[koggdal]

Ah okay 👍

It’s a big complex app, and things take a bit of time to initialize. Not much more though. 5 seconds heartbeat was fine in my testing, 1 second was not. My guess is that it takes up to a couple of seconds or so.

The duplicate issue you linked to does seem similar in the sense of us also having multiple configurations exported from the main configuration. One of them is a dev server configuration that is there to start up the dev server for serving all the other ones. We have multiple apps on the same domain, with shared UI between the apps, so it makes sense for us to build all apps together. This does work very well for us and has for a long time. The only problem we have is the heartbeat timer that is now disconnecting the client.

So, while I can’t speak for the setup of the author of the other issue, they do seem related. The other issue talks about building libraries used by an app, while this is building multiple apps.

I’m open to either change here:

1. Adding option for increasing timer value.
2. Fix runtime logic for webpack-dev-server.

If you believe the slowness comes from running one dev server instead of one per app, any solution for that case would also work! 👍 I do believe however that 1 second can be optimistic to work for all project sizes on all computers under varying load. So even with something else, providing an option for changing that timer value could be useful.

If you prefer to move the conversation to the other issue, that’s fine too. I just wanted to make sure the details in this issue were not lost, as the other issue didn’t mention anything about timing at all.

Thank you!

[alexander-akait]

Will be great if you provide steps to reproduce, it will allow to fix problem very fast

[alexander-akait]

Sorry, tried different solutions and can't reproduce infinity loop, please provide steps to reproduce and I will reopen

[alexander-akait]

Maybe problem html-webpack-harddisk-plugin, maybe something write invalid hash, maybe something was changed and changed again, a lot of possible problems

[koggdal]

Thanks for testing! 🙏

I put together a small reproducible test case here: https://github.com/koggdal/webpack-dev-server-issue-3831-demo

As mentioned, this issue is similar to the other issue you linked (maybe even the same), and I mainly wanted to bring the attention to the details covered in this issue around the commit that started the issues.

It does seem to be related to the multiple configuration setup, as the reload seems to happen more often when more configs are added in the entry file (webpack/entry.config.js).

What's interesting is that changing the heartbeat timer value has no effect now that I'm trying with this test repo (it did fix it every time I tested it in our real setup though). So I'm not sure what the involvement of that timer is for this issue, but maybe it can affect something.

If this is in fact not really caused by that heartbeat timer, do you see what it could be caused by, and how it could be fixed (on your side or our side)?

[alexander-akait]

Give me time, I will look

[alexander-akait]

I put together a small reproducible test case here: https://github.com/koggdal/webpack-dev-server-issue-3831-demo

404, maybe private?

[koggdal]

Oh yes, sorry about that! Changed to public now 👍

[alexander-akait]

Interesting, dev server is thinking it is no initial build due different values __webpack_hash__ and try to reload, when do it again, because it is different again and again and again

[alexander-akait]

Does your build depended, i.e. dev-server.config depend on other configuration? You need to use https://webpack.js.org/configuration/other-options/#dependencies

[alexander-akait]

And as I understand server config doesn't have page, you need working only on product-* pages, right?

[alexander-akait]

And yes, it is duplicate of #3803, shorty - let's take for example product-a we use __webpack_hash__ (special webpack variable to get hash of build) to determine the initial value of hash of build, so if you reload page and change something we can understand if something has changed since the last time you open page, but in your case you send hash of dev-server.config build because only there dev server was started, and when you change something we rebuild all what needed and then send hash of dev-server.config build, but product-a and dev-server.config hashes are always different, previously, this did not lead to a problem for v3, but in fact you still had a broken reload logic, your mix hashes of builds.

Three solutions:

1. start dev server for each configuration, i.e. multiple dev server mode (we are working on it)
2. wait fix for multiple web targets support here Browser reloads infinitely when using multi-compiler mode #3803 (your case), but again, you always send a hash of devServer configuration, and you need to set dependencies: ['product-a', 'product-b', 'product-c', 'product-d', 'product-e', 'product-f', 'product-g'], for dev-server.config
3. migrate on multi entries configuration - it will open up more opportunities for optimization

[alexander-akait]

Let's close in favor #3803, because multiple dev server is not good choice in your case, feel free to feedback here and in #3803