Inconsistent (JS) Output Hashes with emitted CSS files

[lukeed]

• Rollup Version: 2.23.0
• Operating System (or Browser): macOS
• Node Version (if applicable): 10.x - 14.6
• Link to reproduction: https://github.com/lukeed/bug-rollup-css-hashes

Expected Behavior

The generated file hashes should be identical between builds since nothing is changing.

Actual Behavior

Within ~10 builds of the same source files, there will be files that have different build hashes despite having identical output content – or in most cases, identical content except for import statements to other files that also inexplicably change.

In the repro provided (please see readme), there are two files that will have different output hashes within a 10-build loop. They have a maximum of 2 hash variations per file, which I find odd.

It seems to happen with a particular combination of shared dependency trees, dynamic imports, and CSS files (emitted as assets). I dumbed it down considerably, but it's by no means a farfetched application setup.

Thanks~!

[developit]

I believe this may be #3415 (see https://bundlers.tooling.report/hashing/asset-cascade/#rollup).

[lukeed]

Nothing changes though so I didn't think it was related

[lukeed]

I've also spent some time on augmentChunkHash to no avail. The issue persisted here with or without it. I left it out since it wasn't necessary for reproduction

[lukeed]

Additional things I have tried:

• Moving the demo/styles plugin to use a transform hook instead of load hook
• Emitting the CSS file with/without a name field
• Emitting the CSS file with/without the moduleSideEffects field (and with "no-treeshake")
• Adding various augmentChunkHash impls in attempt to force a consistent rehash (including reading from source files)
• Using resolveFileUrl to enforce resolution from an output path

They all have no effect.

It feels like there's an async step that isn't properly awaited somewhere; and so some builds do get to use that info-block as part of hash calculation, while others don't/aren't aware of that extra bit of information.

[benmccann]

This reproduces with older dependencies in the sample project, so it doesn't seem like a recent regression:

  "devDependencies": {
    "@rollup/plugin-node-resolve": "6.0.0",
    "rollup": "2.0.0"
  },

I also tried making the load implementation of the plugin synchronous and it still happened as well

[lukeed]

@benmccann thank you 🙏 I was going to walk through Rollup version history tomorrow

[lukastaegert]

My guess is that it is a race condition that depends on the order in which your modules resolve, load or transform. Would need a little more digging but if that is the case, it might be solvable by manually sorting some internal data structures once the graph is complete.

[lukeed]

That sounds like it could fit what I'm experiencing. Thank you for taking a look :)

[lukastaegert]

It turns out the issue stems from the fact that two assets with the same name "index" are emitted. When an asset is emitted, Rollup assigns a reference id to that asset. To make things reproducible, this id is based on a hash of the asset name. In case there are two assets with the same name, the second one of course will receive a different reference id, however it depends on the load order which is the second one. I believe the only way to make this more stable is to add information about how the file was emitted to the hash of the reference id, let's see.

[lukastaegert]

And of course the reference if is relevant because at the time when the hashing happens, the import.meta.ROLLUP_FILE_URL_<xyz> is still in the code. A new algorithm would also fix this. But I think it does not hurt either way to make the reference ids more reproducible.

[lukastaegert]

Actually it turns out this is quite tricky to fix as the plugin context at the moment does not "know" from what hook and associated with what file id it is called. Although this is not pretty, I would rather hope to fix this by fixing the hashing altogether.

[lukeed]

So if I understand correctly, you're saying that the internal reference IDs are different despite pointing to the same emitted asset? Even though those referenced files are built consistently, each of the reference importers are assuming/working with a different reference value. Because of this, the importers are generated with varying hash outputs?

// Foo/index.css
//=> index.1234.css

// Hello/index.js
import style from '../Foo/index.css';
//=> import.meta.ROLLUP_FILE_URL_123 
//=> index.1234.js

// World/index.js
import style from '../Foo/index.css';
//=> import.meta.ROLLUP_FILE_URL_456 
//=> index.4567.js

Assuming that's true – I'm still not sure how/why that would differ between builds. Something is non-deterministic (or a race condition --> not considered at time of determination)

[lukastaegert]

Well, the non-determinism is that the order in which the transform hooks for Hello and World are processed is not deterministic, it depends on the file system etc. The first one to be processed gets the first id and the second one the second. The problem is that for various reasons, we cannot always return the same reference id for the same file because initially we usually do not know which files are the "same":

• assets do not need to have their sources set
• chunks need their id to be resolved first, which is async

Still, we could try to at least use the same id for assets that have the same source when initially emitted, hm.

[lukeed]

What happens when the emitted name is undefined? How is an identifier assigned?
Do you think the problem will resolve if name were the full source path? Or does that extra information get lost anyway? (sorry I wish i were more familiar with internals right about now)

[lukastaegert]

When there is neither a name nor fileName, there is a fixed sequence of ids that is used, i.e. the first without a name always gets the same id. I do not think you can use the full path in the name as it cannot be an absolute path, and if it contains slashes, the file will be put into a sub-folder. Really tricky.

[lukeed]

Gotcha, okay. Because I've tried without name and fileName fields and nothing changed.

[lukastaegert]

Should be finally fixed with #4543

[rollup-bot]

This issue has been resolved via #4543 as part of rollup@3.0.0-7. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-7 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4543 as part of rollup@3.0.0-8. Note that this is a pre-release, so to test it, you need to install Rollup via npm install rollup@3.0.0-8 or npm install rollup@beta. It will likely become part of a regular release later.

[rollup-bot]

This issue has been resolved via #4543 as part of rollup@3.0.0. You can test it via npm install rollup.