Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use stream#pipeline instead of stream.pipe #8235

Merged
merged 2 commits into from Jun 21, 2022
Merged

Use stream#pipeline instead of stream.pipe #8235

merged 2 commits into from Jun 21, 2022

Conversation

mischnic
Copy link
Member

@mischnic mischnic commented Jun 20, 2022

Fixes No such file: "../.parcel-cache/283921823"
A regression from #8194

Apparently this call would still return even though the stream wasn't fully written yet (and therefore the file wasn't renamed):

    await new Promise((resolve, reject) => {
      stream
        .pipe(this.fs.createWriteStream(this._getCachePath(`${key}-large`)))
        .on('error', reject)
        .on('finish', resolve);
    });

Using stream.pipeline works (which also has official promise capabilities).

Tested in a project that would previously fail deterministically with that error on every build. Now I don't see the error anymore

@parcel-benchmark
Copy link

Benchmark Results

Kitchen Sink ✅

Timings

Description Time Difference
Cold 1.50s -38.00ms
Cached 338.00ms -12.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 74.00ms -8.00ms 🚀
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 75.00ms -8.00ms 🚀
dist/modern/parcel.7cdb0fad.webp 102.94kb +0.00b 76.00ms -7.00ms 🚀
dist/legacy/index.2c76ad23.js 1.66kb +0.00b 424.00ms -36.00ms 🚀
dist/legacy/index.8aaa89c9.js 1.20kb +0.00b 423.00ms -38.00ms 🚀
dist/modern/index.6be20f01.js 1.13kb +0.00b 423.00ms -37.00ms 🚀
dist/legacy/index.b8ae99ba.css 94.00b +0.00b 242.00ms -13.00ms 🚀
dist/modern/index.31cedca9.css 94.00b +0.00b 241.00ms -14.00ms 🚀

Cached Bundles

Bundle Size Difference Time Difference
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 242.00ms +165.00ms ⚠️
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 243.00ms +165.00ms ⚠️
dist/modern/parcel.7cdb0fad.webp 102.94kb +0.00b 243.00ms +164.00ms ⚠️
dist/legacy/index.2c76ad23.js 1.66kb +0.00b 441.00ms -34.00ms 🚀
dist/legacy/index.8aaa89c9.js 1.20kb +0.00b 441.00ms -34.00ms 🚀
dist/modern/index.6be20f01.js 1.13kb +0.00b 441.00ms -35.00ms 🚀

React HackerNews ✅

Timings

Description Time Difference
Cold 9.43s +9.00ms
Cached 463.00ms +11.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

AtlasKit Editor ✅

Timings

Description Time Difference
Cold 1.70m +749.00ms
Cached 2.73s +53.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/esm.c7dc1640.js 61.96kb +0.00b 1.27m +34.67s ⚠️
dist/workerHasher.e50d242f.js 1.72kb +0.00b 1.27m +34.67s ⚠️
dist/16.1969624f.js 1.08kb +0.00b 41.03s +3.73s ⚠️
dist/16.069344b7.js 905.00b +0.00b 41.03s +3.73s ⚠️
dist/simpleHasher.46d6f2e5.js 742.00b +0.00b 1.27m +34.67s ⚠️
dist/index.html 240.00b +0.00b 1.27m +43.12s ⚠️

Cached Bundles

Bundle Size Difference Time Difference
dist/esm.c7dc1640.js 61.96kb +0.00b 41.88s -34.20s 🚀
dist/workerHasher.e50d242f.js 1.72kb +0.00b 41.88s -34.20s 🚀
dist/16.1969624f.js 1.08kb +0.00b 37.77s -2.79s 🚀
dist/16.069344b7.js 905.00b +0.00b 37.77s -2.79s 🚀
dist/simpleHasher.46d6f2e5.js 742.00b +0.00b 41.88s -34.20s 🚀
dist/index.html 240.00b +0.00b 33.17s -42.94s 🚀

Three.js ✅

Timings

Description Time Difference
Cold 6.91s -3.00ms
Cached 279.00ms -3.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

Click here to view a detailed benchmark overview.

@Cristy94
Copy link

@mischnic

Apparently this call would still return even though the stream wasn't fully written yet (and therefore the file wasn't renamed)

Could you describe in short why this is the case?
Wouldn't it only return in those two cases?

        .on('error', reject)
        .on('finish', resolve);

@mischnic
Copy link
Member Author

The only explanation I have is that .on('finish', resolve) was triggered too early (and I guess somehow called on the wrong stream? maybe it finished reading but finished writing).

@Cristy94
Copy link

createWriteStream

Hmm, maybe close was the right event to listen too? https://nodejs.org/api/fs.html#filehandlecreatewritestreamoptions
Even though by default it seems like close is called anyway after finish:

If autoClose is set to true (default behavior) on 'error' or 'finish' the file descriptor will be closed automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants