Improve compile time on large application #50792
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timneutkens
requested review from
ijjk,
shuding,
huozhi and
feedthejim
as code owners
|
Current dependencies on/for this PR:
This comment was auto-generated by Graphite. |
huozhi
previously approved these changes
Jun 5, 2023
|
Allow CI Workflow Run
Note: this should only be enabled once the PR is ready to go and can only be enabled by a maintainer |
feedthejim
previously approved these changes
Jun 5, 2023
timneutkens
force-pushed
the
06-05-Improve_compile_time_on_large_application
branch
from
a57540d
to
4ea9575
Compare
1 task
1 task
hydRAnger
pushed a commit
to hydRAnger/next.js
that referenced
this pull request
Jun 12, 2023
## What? While investigating slow compilation for a page on vercel.com in development I found that there was close to 10 seconds of time unaccounted for in `.next/trace`. Ran a profile and found that time was spent in watchpack `batch`, specifically calling `close` many times. When I tried to debug this further by running unbundled webpack I noticed the same issue didn't exists. ### Before <img width="1329" alt="Before" src="https://github.com/vercel/next.js/assets/6324199/9ace4628-db04-4de7-993f-65aef9dffc55"> ### After <img width="1278" alt="After" src="https://github.com/vercel/next.js/assets/6324199/55d5e58b-4a27-4235-8dea-723a7a78c117"> ## Raw numbers <table> <tr> <td>Before</td> <td>After</td> <td>Delta</td> <td>Delta (percent)</td> </tr> <tr> <td>13840 ms</td> <td>3580 ms</td> <td>-10260 ms</td> <td>-74.13%</td> </tr> </table> ## How? Investigated further and found that specifically not minifying watchpack solved the issue. <!-- Thanks for opening a PR! Your contribution is much appreciated. To make sure your PR is handled as smoothly as possible we request that you follow the checklist sections below. Choose the right checklist for the change(s) that you're making: ## For Contributors ### Improving Documentation or adding/fixing Examples - The "examples guidelines" are followed from our contributing doc https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md - Make sure the linting passes by running `pnpm build && pnpm lint`. See https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md ### Fixing a bug - Related issues linked using `fixes #number` - Tests added. See: https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ### Adding a feature - Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. (A discussion must be opened, see https://github.com/vercel/next.js/discussions/new?category=ideas) - Related issues/discussions are linked using `fixes #number` - e2e tests added (https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs - Documentation added - Telemetry added. In case of a feature if it's used or not. - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ## For Maintainers - Minimal description (aim for explaining to someone not on the team to understand the PR) - When linking to a Slack thread, you might want to share details of the conclusion - Link both the Linear (Fixes NEXT-xxx) and the GitHub issues - Add review comments if necessary to explain to the reviewer the logic behind a change ### What? ### Why? ### How? Closes NEXT- Fixes # -->
|
We noticed many fs close calls in our app and switched to polling in watchpack(since it does not relies on fs watcher). Polling is still faster (~12s now vs ~20s in watcher) in our app even after this PR's changes. We added this to next.config.js This issue likely appeared in node 14 onwards nodejs/node#29949, not sure how precompiling watchpack is increasing fs call. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update: we found the root cause: #51826
What?
While investigating slow compilation for a page on vercel.com in development I found that there was close to 10 seconds of time unaccounted for in
.next/trace. Ran a profile and found that time was spent in watchpackbatch, specifically callingclosemany times. When I tried to debug this further by running unbundled webpack I noticed the same issue didn't exists.Before
After
Raw numbers
How?
Investigated further and found that specifically not minifying watchpack solved the issue.