Report performance in CI

[TrickyPi]

This PR contains:

• bugfix
• feature
• refactor
• documentation
• other

Are tests included?

• yes (bugfixes and features will not be merged without tests)
• no

Breaking Changes?

• yes (breaking changes will not be merged unless absolutely necessary)
• no

List any relevant issue numbers:

Description

Through this report, we can roughly determine whether there are performance regression or improvement in the PR.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
rollup	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 26, 2024 3:27pm

[github-actions]

Thank you for your contribution! ❤️

You can try out this pull request locally by installing Rollup via

npm install rollup/rollup#xiaopi/ci-performance-report

Notice: Ensure you have installed Rust nightly. If you haven't installed it yet, please first see https://www.rust-lang.org/tools/install to learn how to download Rustup and install Rust, then see https://rust-lang.github.io/rustup/concepts/channels.html to learn how to install Rust nightly.

or load it into the REPL:
https://rollup-qwqal0kbf-rollup-js.vercel.app/repl/?pr=5488

[github-actions]

Performance report!

Rough benchmark

Command	Mean [s]	Min [s]	Max [s]	Relative
node _benchmark/previous/bin/rollup -i ./perf/entry.js -o _benchmark/result/previous.js	9.444 ± 0.097	9.333	9.511	1.00
node _benchmark/current/bin/rollup -i ./perf/entry.js -o _benchmark/result/current.js	9.470 ± 0.109	9.345	9.548	1.00 ± 0.02

Internal benchmark

BUILD: 8442ms (-33ms, -0.4%), 743 MB

initialize: 0ms, 24.3 MB (+8%)

generate module graph: 3291ms (+18ms, +0.6%), 565 MB

generate ast: 1554ms (+38ms, +2.5%), 557 MB

sort and bind modules: 451ms, 606 MB

mark included statements: 4681ms (-56ms, -1.2%), 743 MB

treeshaking pass 1: 1573ms (-27ms, -1.7%), 706 MB
treeshaking pass 2: 756ms (-17ms, -2.2%), 728 MB
treeshaking pass 3: 299ms, 731 MB
treeshaking pass 4: 283ms, 733 MB
treeshaking pass 5: 326ms, 739 MB
treeshaking pass 6: 267ms, 741 MB
treeshaking pass 7: 250ms, 740 MB
treeshaking pass 8: 245ms, 741 MB
treeshaking pass 9: 220ms, 742 MB
treeshaking pass 10: 222ms, 747 MB
treeshaking pass 11: 217ms, 743 MB

GENERATE: 909ms, 1.02 GB

initialize render: 0ms, 906 MB

generate chunks: 87ms, 919 MB

optimize chunks: 0ms, 907 MB

render chunks: 803ms, 993 MB

transform chunks: 17ms, 1.02 GB

generate bundle: 0ms, 1.02 GB

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.81%. Comparing base (778066d) to head (5e558f0).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5488   +/-   ##
=======================================
  Coverage   98.81%   98.81%           
=======================================
  Files         238      238           
  Lines        9451     9451           
  Branches     2408     2408           
=======================================
  Hits         9339     9339           
  Misses         47       47           
  Partials       65       65           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lukastaegert]

This is amazing! One question, though: Currently, this is using the Rollup build, which has many plugins that are out of our control etc. and not too close to the core Rollup performance. I recently added a benchmarking script that you get when you run npm run perf (replacing an older benchmark that did not work too well...).
What this does is basically run a benchmark used by esbuild: Download the three.js sources and bundle them ten times. This does not require any plugins and is thus a much better measure of bundling performance. It will also report detailed statistics in which parts of the application we gain or lose performance as well as where the memory consumption improved or became worse. Not sure if it is easy to put this into a report, but this is something I thought we should definitely have eventually.

[TrickyPi]

I recently added a benchmarking script that you get when you run npm run perf

👍, Oh, I see!

Not sure if it is easy to put this into a report, but this is something I thought we should definitely have eventually.

Yeah, I agree with you! I think it's not hard to include this information in the report. I'll have a look at it!

[lukastaegert]

Nice 👍

One thing to consider if we want to save a runner could be to not build the artifacts in the job but move the report phase to the other Github workflow where we already build all artifacts and run it similar to the tests and smoke tests. But on the other hand, it does not make much of a difference.

[github-actions]

This PR has been released as part of rollup@4.17.0. You can test it via npm install rollup.