Multiple DefinePlugin instances cause aggressive cache invalidation due to collisions

[issacgerges]

Bug report

When multiple instances of DefinePlugin are inserted into the compilation, constants unkeyed by an instance identifier (specifically VALUE_DEP_PREFIX and VALUE_DEP_MAIN) cause inconsistent hash values causing massive cache invalidations (in our case making webpack believe 90% of modules had changed on any file change).

Although its uncommon for developers to use multiple instances of DefinePlugin, some plugins like webpack-plugin-serve automatically inject their own instances.

What is the current behavior?

Multiple instances cause hash inconsistency causing all modules to be considered changed

What is the expected behavior?

The DefinePlugin hash should only change if the defineplugin inputs change

Other relevant information:
webpack version: 5.90.3
Node.js version: v20.11.1
Operating System: OSX
Additional tools:

[alexander-akait]

Please create reproducible test repo, because I can't reproduce it, also webpack-plugin-serve is deprecated, please don't use it

[alexander-akait]

Oh, the problem is here - https://github.com/shellscape/webpack-plugin-serve/blob/master/lib/index.js#L168, it is always a unique value, so webpack invalidates caches between multiple serve commands, we can't fix it here, sorry

[issacgerges]

Interesting, will dig in a bit more. That said the following patch does seem to fix my issue

diff --git a/lib/DefinePlugin.js b/lib/DefinePlugin.js
index ed02a3cbd2d5d2c49aa2d49bc1732ab6b47ea004..27a49fa9e649f7111118031fd54185373c169766 100644
--- a/lib/DefinePlugin.js
+++ b/lib/DefinePlugin.js
@@ -306,11 +306,12 @@ const toCacheVersion = code => {
 
 const PLUGIN_NAME = "DefinePlugin";
 const VALUE_DEP_PREFIX = `webpack/${PLUGIN_NAME} `;
-const VALUE_DEP_MAIN = `webpack/${PLUGIN_NAME}_hash`;
 const TYPEOF_OPERATOR_REGEXP = /^typeof\s+/;
 const WEBPACK_REQUIRE_FUNCTION_REGEXP = /__webpack_require__\s*(!?\.)/;
 const WEBPACK_REQUIRE_IDENTIFIER_REGEXP = /__webpack_require__/;
 
+let instanceCount = 1;
+
 class DefinePlugin {
 	/**
 	 * Create a new define plugin
@@ -318,6 +319,10 @@ class DefinePlugin {
 	 */
 	constructor(definitions) {
 		this.definitions = definitions;
+		// patch support multiple instances
+		this.value_dep_prefix = `webpack/${PLUGIN_NAME}-${instanceCount} `;
+		this.value_dep_main = `webpack/${PLUGIN_NAME}-${instanceCount}_hash`;
+		instanceCount += 1;
 	}
 
 	/**
@@ -349,7 +354,7 @@ class DefinePlugin {
 				const mainHash = createHash(compilation.outputOptions.hashFunction);
 				mainHash.update(
 					/** @type {string} */ (
-						compilation.valueCacheVersions.get(VALUE_DEP_MAIN)
+						compilation.valueCacheVersions.get(this.value_dep_main)
 					) || ""
 				);
 
@@ -359,14 +364,14 @@ class DefinePlugin {
 				 * @returns {void}
 				 */
 				const handler = parser => {
-					const mainValue = compilation.valueCacheVersions.get(VALUE_DEP_MAIN);
+					const mainValue = compilation.valueCacheVersions.get(this.value_dep_main);
 					parser.hooks.program.tap(PLUGIN_NAME, () => {
 						const buildInfo = /** @type {BuildInfo} */ (
 							parser.state.module.buildInfo
 						);
 						if (!buildInfo.valueDependencies)
 							buildInfo.valueDependencies = new Map();
-						buildInfo.valueDependencies.set(VALUE_DEP_MAIN, mainValue);
+						buildInfo.valueDependencies.set(this.value_dep_main, mainValue);
 					});
 
 					/**
@@ -377,8 +382,8 @@ class DefinePlugin {
 							parser.state.module.buildInfo
 						);
 						buildInfo.valueDependencies.set(
-							VALUE_DEP_PREFIX + key,
-							compilation.valueCacheVersions.get(VALUE_DEP_PREFIX + key)
+							this.value_dep_prefix + key,
+							compilation.valueCacheVersions.get(this.value_dep_prefix + key)
 						);
 					};
 
@@ -648,7 +653,7 @@ class DefinePlugin {
 					Object.keys(definitions).forEach(key => {
 						const code = definitions[key];
 						const version = toCacheVersion(code);
-						const name = VALUE_DEP_PREFIX + prefix + key;
+						const name = this.value_dep_prefix + prefix + key;
 						mainHash.update("|" + prefix + key);
 						const oldVersion = compilation.valueCacheVersions.get(name);
 						if (oldVersion === undefined) {
@@ -678,7 +683,7 @@ class DefinePlugin {
 				walkDefinitionsForValues(definitions, "");
 
 				compilation.valueCacheVersions.set(
-					VALUE_DEP_MAIN,
+					this.value_dep_main,
 					/** @type {string} */ (mainHash.digest("hex").slice(0, 8))
 				);
 			}

[alexander-akait]

hm, we use mainHash.update("|" + prefix + key);, so each value should be uniuq, can you use console.log() for https://github.com/webpack/webpack/blob/main/lib/DefinePlugin.js#L322 and run build, so we will get all values and I will look, if you have something secret just change it for *** for me

[issacgerges]

Will do. I believe the issue was the value is stash into a map (on compiler I believe) that uses only the DefinePlugin_hash key (which has no uniqueness)

[issacgerges]

webpack-plugin-serve uses a stringified json that is stable across invocations. The {stable json} value below is the same string (of length 9593)

// invocation one
{
  __TOKEN_ONE__: undefined,
  __TOKEN_TWO__: undefined,
  __CACHE_KEY__: '"string"',
  'process.env': { NODE_ENV: '"development"' }
}
// invocation two
{ 'process.env.NODE_ENV': '"development"' }
// invocation three
{
  'ʎɐɹɔosǝʌɹǝs': '{stable json}'
}

// invocation four (on incremental change)
{
  'ʎɐɹɔosǝʌɹǝs': '{stable json}'
}

[alexander-akait]

Can you try to remove 'process.env': { NODE_ENV: '"development"' } from the first invocation?

[Hazeleyed88]

Cam u email me I do not know how to even access assets I need some truthful and trustworthy to help will u

[issacgerges]

I'll close this issue. In the end it appears the cause by webpack-plugin-serve injecting a new instance of DefinePlugin on every watch run. I don't think this is standard enough to require any support on webpacks end. I was able to patch around it and am in the process of migrating to webpack-dev-server

https://github.com/shellscape/webpack-plugin-serve/blob/b19fce1db5f881af603879b8bb9c555c16ae394f/lib/index.js#L266-L268