Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I improve the build speed of node js? #35888

Open
TheEpicFace007 opened this issue Oct 30, 2020 · 16 comments
Open

How can I improve the build speed of node js? #35888

TheEpicFace007 opened this issue Oct 30, 2020 · 16 comments
Labels
build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.

Comments

@TheEpicFace007
Copy link

I was playing with the code source of node however I find the build speed really slow(1h30m). Is there any way I can improove the build speed of node js?
My compiler is msvc

@Trott Trott added build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform. labels Oct 30, 2020
@bzoz
Copy link
Contributor

bzoz commented Nov 2, 2020

Hi, thanks for the report.

Sorry, but you will just need a beefier PC to build Node faster. A good multi-core CPU and an NVME drive will do wonders. The first, clean build is exceptionally hard, subsequent ones should be faster though.

That said, there is indeed a regression in Node build files introduced in #32698. You can try building this commit (one before the change): b79829c. Depending on your configuration, it should be slightly faster.

@bzoz
Copy link
Contributor

bzoz commented Nov 2, 2020

Opened an issue with this regression: #35921, @ViniDalvino thanks for bringing this up to our attention!

@tniessen
Copy link
Member

tniessen commented Nov 4, 2020

In the long term, we will probably switch to a better build system, see nodejs/TSC#901. Unfortunately, building Node.js with its dependencies is super slow, especially on Windows. As @bzoz wrote, only the first build should take hours, and subsequent incremental builds should complete within minutes.

@TheEpicFace007
Copy link
Author

Does the issue might be due to me building node exclusively with the build script(vcbuild.bat)? @tniessen @bzoz

@devsnek
Copy link
Member

devsnek commented Nov 4, 2020

@ViniDalvino out of curiosity are you using the -j flag when you build? for example if your cpu has 8 logical cores, you can do make -j8.

@TheEpicFace007
Copy link
Author

@devsnek it's not a valid command line option. I think msvc multithread the build automatically anyway

@bzoz
Copy link
Contributor

bzoz commented Nov 5, 2020

Yeah, vcbuild.bat does not use -j, it should use the max amount of cores by default. There is this regression, where for some targets it would not do that, I've made #35932 to fix it.

That said, @ViniDalvino you will still need a better PC. I have a hunch that even with this change it will take ~1h to build Node on your box - you can test that by checking out my PR code and running vcbuild.

@tniessen
Copy link
Member

tniessen commented Nov 5, 2020

That said, @ViniDalvino you will still need a better PC. I have a hunch that even with this change it will take ~1h to build Node on your box - you can test that by checking out my PR code and running vcbuild.

That should only be true for a complete rebuild, and incremental builds should be much faster. If incremental builds take ~1h on any modern hardware, that's on us or on the build tools.

@ViniDalvino I have developed on terribly slow machines in the past, and based on my own experience, I would suggest considering using Linux for Node.js development. I developed Node.js on Windows for a long time, but it cannot really compete with development on Linux for me. I should probably just buy a better computer as suggested above, but things are much smoother on Linux than on Windows.

@TheEpicFace007
Copy link
Author

TheEpicFace007 commented Nov 5, 2020

Would node need to be built through the solution in visual studio in order to be quick? What I mean is that if you compile node through visual studio you can select what you compile; As example don't have to compile v8 everytime.

@tniessen
Copy link
Member

tniessen commented Nov 5, 2020

You can try and use the Windows task manager to check CPU and memory utilization. CPU utilization should be close to 100% during compilation if everything is working correctly.

@TheEpicFace007
Copy link
Author

@tniessen The CPU usage seem fine.
![
image
](CPU Usage)

@gengjiawen
Copy link
Member

gengjiawen commented Nov 12, 2020

Any chance we can bring ninja on windows ? cc @nodejs/gyp

The increment build on windows still unbearable.

@ghost
Copy link

ghost commented Jan 7, 2022

Sorry for the necrobump, I'm compiling nodejs on Gentoo linux and seriously, it takes looong time and lots of ram to compile. I've ended using ccache to speed it up because on my laptop it takes up to 2 hours (yes, my laptop is a complete pain). Do you have any other suggestion too?

@TheEpicFace007
Copy link
Author

I don't have any but thanks for suggesting ccache for improving build speed @scardracs +1

@aleclarson
Copy link
Contributor

Is there a way to avoid compiling the lib/* modules into the binary, so I can tweak/debug the JS internals without a slow feedback loop?

@mmarchini
Copy link
Contributor

@aleclarson Yes, there is a build flag that can be used for that (as per BUILDING.md):

$ ./configure --node-builtin-modules-path $(pwd)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

No branches or pull requests

8 participants