Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in watch mode due to unique child compiler IDs #1835

Closed
helloitsjoe opened this issue Dec 12, 2023 · 1 comment · Fixed by #1836
Closed

Memory leak in watch mode due to unique child compiler IDs #1835

helloitsjoe opened this issue Dec 12, 2023 · 1 comment · Fixed by #1836

Comments

@helloitsjoe
Copy link
Contributor

helloitsjoe commented Dec 12, 2023

Current behaviour 💣

During watch mode, each new child compiler creates a copy of source strings, which are never removed from memory.

I think I've narrowed this down to this commit aa64b82 where HtmlWebpackPlugin started using a unique ID for each child compiler instead of the template path. This cache in MemoryCachePlugin uses the unique ID when storing an entry, so the cache entries grow with each compilation.

I don't see any mention of the change in the changelog or PR description (😄) so I'm not sure why the change was made, but id doesn't seem to be used anywhere else in HtmlWebpackPlugin. Let me know if there's an explanation I'm missing somewhere!

Potential fix 🛠️

The memory leak goes away when aa64b82 is reverted to use template, and unit tests still pass. Is it possible to revert this change, or if it's solving a problem, to implement it in a different way? I'm happy to help.

Screenshots

Duplicated strings in memory

The size of the leak is small in this example, but the reproduction repo below shows an example of a much larger leak.
Screenshot 2023-12-11 at 10 08 47 PM

MemoryCachePlugin cache

Note the HtmlWebpackPlugin_0-15 - this is the 15th iteration i.e. 15th child compiler.
Screenshot 2023-12-11 at 10 12 58 PM

Expected behaviour ☀️

Source strings are cleaned up (or able to be garbage collected) between recompilations in watch mode

Reproduction Example 👾

The main branch of this repo includes a custom loader that makes the leak obvious: https://github.com/helloitsjoe/webpack-cache-memory-leak

But the leak is also visible on this simplified branch with vanilla use of HtmlWebpackPlugin: https://github.com/helloitsjoe/webpack-cache-memory-leak/tree/simplified

Environment 🖥

Node.js v20.5.1
darwin 22.5.0
npm 9.8.0
webpack@5.89.0
html-webpack-plugin@5.5.4
@helloitsjoe
Copy link
Contributor Author

I've created a pull request here: #1836

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant