Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add included metadata to chunks to allow deduplicate chunk loads at runtime #4488

Merged
merged 11 commits into from
Apr 17, 2023

Conversation

sokra
Copy link
Member

@sokra sokra commented Apr 6, 2023

Description

Chunk loading is now more than just a single chunk filepath. It might consist of additional metadata which tells the chunk loading logic the entry modules included in this chunk. This allows the runtime logic to avoid loading chunks when all entry modules are already loaded.

This means loading the same chunk from multiple sources (with different available modules) will only trigger a single load and so avoids transferring more code than needed.

This is especially relevant in app dir where every client component is considers as separate chunk loading.

next.js: vercel/next.js#48025

@vercel
Copy link

vercel bot commented Apr 6, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-native-web 🔄 Building (Inspect) Apr 17, 2023 9:42am
examples-tailwind-web 🔄 Building (Inspect) Apr 17, 2023 9:42am
turbo-site ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 17, 2023 9:42am
8 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-cra-web ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-designsystem-docs ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-gatsby-web ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-kitchensink-blog ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-nonmonorepo ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-svelte-web ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am
examples-vite-web ⬜️ Ignored (Inspect) Apr 17, 2023 9:42am

@github-actions
Copy link
Contributor

github-actions bot commented Apr 6, 2023

✅ This changes can build next-swc

@github-actions
Copy link
Contributor

github-actions bot commented Apr 6, 2023

🟢 CI successful 🟢

Thanks

@github-actions
Copy link
Contributor

Benchmark for 0ba9742

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8903.64µs ± 69.45µs 8998.55µs ± 56.37µs +1.07%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8003.44µs ± 58.84µs 8047.12µs ± 77.24µs +0.55%
bench_startup/Turbopack CSR/1000 modules 875.81ms ± 1.48ms 883.76ms ± 4.23ms +0.91%

@sokra sokra marked this pull request as ready for review April 16, 2023 05:02
@sokra sokra requested a review from a team as a code owner April 16, 2023 05:02
@sokra sokra requested a review from alexkirsz April 16, 2023 05:03
@github-actions
Copy link
Contributor

Benchmark for b245f99

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 9769.92µs ± 55.45µs 9720.19µs ± 83.61µs -0.51%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8657.45µs ± 52.53µs 8764.49µs ± 157.53µs +1.24%
bench_startup/Turbopack CSR/1000 modules 885.77ms ± 1.64ms 888.78ms ± 6.84ms +0.34%

crates/turbopack-core/src/chunk/mod.rs Show resolved Hide resolved
crates/turbopack-dev/js/src/runtime.dom.js Outdated Show resolved Hide resolved
crates/turbopack-dev/js/src/runtime.dom.js Outdated Show resolved Hide resolved
crates/turbopack-dev/js/src/runtime.dom.js Outdated Show resolved Hide resolved
crates/turbopack-dev/js/src/runtime.js Outdated Show resolved Hide resolved
loadChunk(
typeof otherChunkData === "string"
? otherChunkData
: otherChunkData.path,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be a function getChunkPath(chunkData), we do this very often.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only used once in this file

crates/turbopack-dev/src/ecmascript/chunk_data.rs Outdated Show resolved Hide resolved
crates/turbopack-dev/src/ecmascript/manifest/chunk_item.rs Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

Benchmark for db28da1

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8994.33µs ± 33.28µs 8900.39µs ± 50.00µs -1.04%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8110.44µs ± 23.99µs 8303.01µs ± 258.58µs +2.37%
bench_startup/Turbopack CSR/1000 modules 887.47ms ± 1.12ms 885.67ms ± 5.83ms -0.20%

@github-actions
Copy link
Contributor

Benchmark for 729bc00

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8922.81µs ± 46.60µs 8895.42µs ± 54.19µs -0.31%
bench_hmr_to_eval/Turbopack CSR/1000 modules 7972.19µs ± 48.16µs 8110.27µs ± 213.30µs +1.73%
bench_startup/Turbopack CSR/1000 modules 860.29ms ± 1.93ms 862.88ms ± 3.32ms +0.30%

@sokra sokra requested a review from alexkirsz April 17, 2023 10:05
@sokra sokra merged commit ab676c5 into main Apr 17, 2023
30 checks passed
@sokra sokra deleted the sokra/runtime-deduplication branch April 17, 2023 11:35
@sokra sokra mentioned this pull request Apr 17, 2023
sokra added a commit to vercel/next.js that referenced this pull request Apr 17, 2023
### What?

* fix app dir chunking
* fix app dir 404s
* improve app dir performance
* rerender shadowportal on errors to re-add nextjs-portal to avoid empty
page
* inject polyfills before user code
* fix manifest generation

see also vercel/turbo#4488

### Why?

App dir was very slow and lead to 404 errors

### How?

add included metadata to chunks to allow deduplicate chunk loads at
runtime
sokra added a commit that referenced this pull request Apr 19, 2023
### Description

follow up changes from #4488
NicholasLYang pushed a commit to NicholasLYang/turbo that referenced this pull request Apr 21, 2023
…untime (vercel#4488)

### Description

Chunk loading is now more than just a single chunk filepath. It might
consist of additional metadata which tells the chunk loading logic the
entry modules included in this chunk. This allows the runtime logic to
avoid loading chunks when all entry modules are already loaded.

This means loading the same chunk from multiple sources (with different
available modules) will only trigger a single load and so avoids
transferring more code than needed.

This is especially relevant in app dir where every client component is
considers as separate chunk loading.

next.js: vercel/next.js#48025
NicholasLYang pushed a commit to NicholasLYang/turbo that referenced this pull request Apr 21, 2023
### Description

follow up changes from vercel#4488
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants