Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consolidate benchmark workflow steps into a single reusable workflow #3782

Merged
merged 1 commit into from Oct 28, 2022
Merged

Consolidate benchmark workflow steps into a single reusable workflow #3782

merged 1 commit into from Oct 28, 2022

Conversation

andrewiggins
Copy link
Member

  • Simplify the definition of benchmarks by creating a new workflow, run-bench.yml, that can be reused for all benchmarks
  • Reuse the ci.yml workflow to build the repo
  • Add trace option to run-bench workflow

@github-actions
Copy link

github-actions bot commented Oct 27, 2022 •
edited

📊 Tachometer Benchmark Results

Summary

duration

  • 02_replace1k: unsure 🔍 -15% - +12% (-13.70ms - +10.65ms)
    preact-local vs preact-master
  • 03_update10th1k_x16: unsure 🔍 -3% - +4% (-1.78ms - +2.03ms)
    preact-local vs preact-master
  • 07_create10k: unsure 🔍 -1% - +1% (-9.42ms - +9.17ms)
    preact-local vs preact-master
  • filter_list: unsure 🔍 -2% - +2% (-0.44ms - +0.57ms)
    preact-local vs preact-master
  • hydrate1k: unsure 🔍 -0% - +2% (-0.54ms - +2.45ms)
    preact-local vs preact-master
  • many_updates: unsure 🔍 -2% - +4% (-0.61ms - +1.38ms)
    preact-local vs preact-master
  • text_update: unsure 🔍 -1% - +2% (-0.04ms - +0.05ms)
    preact-local vs preact-master
  • todo: unsure 🔍 -2% - +5% (-1.57ms - +3.42ms)
    preact-local vs preact-master

usedJSHeapSize

  • 02_replace1k: unsure 🔍 -7% - +12% (-0.32ms - +0.54ms)
    preact-local vs preact-master
  • 03_update10th1k_x16: unsure 🔍 -0% - +0% (-0.00ms - +0.01ms)
    preact-local vs preact-master
  • 07_create10k: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • filter_list: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • hydrate1k: unsure 🔍 -4% - +4% (-0.53ms - +0.61ms)
    preact-local vs preact-master
  • many_updates: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • text_update: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • todo: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master

Results

02_replace1k

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master83.06ms - 100.04ms-unsure 🔍
-12% - +15%
-10.65ms - +13.70ms		unsure 🔍
-7% - +24%
-5.16ms - +19.88ms
preact-local81.30ms - 98.75msunsure 🔍
-15% - +12%
-13.70ms - +10.65ms		-unsure 🔍
-9% - +23%
-6.85ms - +18.51ms
preact-hooks74.99ms - 93.39msunsure 🔍
-21% - +5%
-19.88ms - +5.16ms		unsure 🔍
-20% - +7%
-18.51ms - +6.85ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master4.38ms - 4.94ms-unsure 🔍
-11% - +7%
-0.54ms - +0.32ms		unsure 🔍
-13% - +4%
-0.63ms - +0.19ms
preact-local4.44ms - 5.09msunsure 🔍
-7% - +12%
-0.32ms - +0.54ms		-unsure 🔍
-11% - +7%
-0.55ms - +0.33ms
preact-hooks4.58ms - 5.18msunsure 🔍
-4% - +14%
-0.19ms - +0.63ms		unsure 🔍
-7% - +12%
-0.33ms - +0.55ms		-

run-warmup-0

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master0.96ms - 1.02ms-unsure 🔍
-1% - +6%
-0.01ms - +0.06ms		unsure 🔍
-1% - +7%
-0.00ms - +0.06ms
preact-local0.95ms - 0.99msunsure 🔍
-6% - +1%
-0.06ms - +0.01ms		-unsure 🔍
-2% - +3%
-0.02ms - +0.03ms
preact-hooks0.94ms - 0.98msunsure 🔍
-6% - +0%
-0.06ms - +0.00ms		unsure 🔍
-3% - +2%
-0.03ms - +0.02ms		-

run-warmup-1

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master0.55ms - 0.78ms-unsure 🔍
-23% - +18%
-0.16ms - +0.12ms		unsure 🔍
-29% - +16%
-0.21ms - +0.12ms
preact-local0.61ms - 0.75msunsure 🔍
-19% - +24%
-0.12ms - +0.16ms		-unsure 🔍
-22% - +15%
-0.16ms - +0.11ms
preact-hooks0.59ms - 0.82msunsure 🔍
-19% - +32%
-0.12ms - +0.21ms		unsure 🔍
-16% - +24%
-0.11ms - +0.16ms		-

run-warmup-2

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master0.38ms - 0.47ms-unsure 🔍
-14% - +12%
-0.06ms - +0.05ms		unsure 🔍
-17% - +10%
-0.08ms - +0.04ms
preact-local0.40ms - 0.46msunsure 🔍
-12% - +14%
-0.05ms - +0.06ms		-unsure 🔍
-14% - +9%
-0.06ms - +0.04ms
preact-hooks0.40ms - 0.48msunsure 🔍
-11% - +18%
-0.04ms - +0.08ms		unsure 🔍
-9% - +15%
-0.04ms - +0.06ms		-

run-warmup-3

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master0.61ms - 1.36ms-slower ❌
16% - 213%
0.13ms - 0.92ms		faster ✔
12% - 68%
0.07ms - 1.25ms
preact-local0.34ms - 0.57msfaster ✔
32% - 75%
0.13ms - 0.92ms		-faster ✔
62% - 83%
0.72ms - 1.65ms
preact-hooks1.19ms - 2.10msunsure 🔍
-12% - +146%
+0.07ms - +1.25ms		slower ❌
125% - 393%
0.72ms - 1.65ms		-

run-warmup-4

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master0.30ms - 0.36ms-unsure 🔍
-5% - +20%
-0.01ms - +0.06ms		unsure 🔍
-14% - +7%
-0.05ms - +0.03ms
preact-local0.28ms - 0.33msunsure 🔍
-18% - +4%
-0.06ms - +0.01ms		-unsure 🔍
-21% - +1%
-0.07ms - +0.00ms
preact-hooks0.31ms - 0.37msunsure 🔍
-8% - +15%
-0.03ms - +0.05ms		unsure 🔍
-2% - +24%
-0.00ms - +0.07ms		-

run-final

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master0.31ms - 0.39ms-unsure 🔍
-15% - +15%
-0.05ms - +0.05ms		unsure 🔍
-13% - +13%
-0.05ms - +0.05ms
preact-local0.31ms - 0.39msunsure 🔍
-15% - +15%
-0.05ms - +0.05ms		-unsure 🔍
-13% - +13%
-0.05ms - +0.05ms
preact-hooks0.33ms - 0.38msunsure 🔍
-13% - +13%
-0.05ms - +0.05ms		unsure 🔍
-13% - +13%
-0.05ms - +0.05ms		-
03_update10th1k_x16

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master49.98ms - 52.83ms-unsure 🔍
-4% - +3%
-2.03ms - +1.78ms		unsure 🔍
-5% - +3%
-2.42ms - +1.75ms
preact-local50.26ms - 52.80msunsure 🔍
-3% - +4%
-1.78ms - +2.03ms		-unsure 🔍
-4% - +3%
-2.19ms - +1.77ms
preact-hooks50.22ms - 53.26msunsure 🔍
-3% - +5%
-1.75ms - +2.42ms		unsure 🔍
-3% - +4%
-1.77ms - +2.19ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master3.78ms - 3.79ms-unsure 🔍
-0% - +0%
-0.01ms - +0.00ms		faster ✔
0% - 1%
0.02ms - 0.03ms
preact-local3.78ms - 3.79msunsure 🔍
-0% - +0%
-0.00ms - +0.01ms		-faster ✔
0% - 1%
0.01ms - 0.02ms
preact-hooks3.80ms - 3.81msslower ❌
0% - 1%
0.02ms - 0.03ms		slower ❌
0% - 1%
0.01ms - 0.02ms		-
07_create10k

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master1280.30ms - 1292.29ms-unsure 🔍
-1% - +1%
-9.17ms - +9.42ms		unsure 🔍
-1% - +0%
-12.46ms - +3.24ms
preact-local1279.06ms - 1293.28msunsure 🔍
-1% - +1%
-9.42ms - +9.17ms		-unsure 🔍
-1% - +0%
-13.47ms - +3.99ms
preact-hooks1285.84ms - 1295.98msunsure 🔍
-0% - +1%
-3.24ms - +12.46ms		unsure 🔍
-0% - +1%
-3.99ms - +13.47ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master25.84ms - 25.84ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms		unsure 🔍
-0% - -0%
-0.02ms - -0.02ms
preact-local25.84ms - 25.84msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-unsure 🔍
-0% - -0%
-0.02ms - -0.02ms
preact-hooks25.86ms - 25.86msunsure 🔍
+0% - +0%
+0.02ms - +0.02ms		unsure 🔍
+0% - +0%
+0.02ms - +0.02ms		-
filter_list

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master22.90ms - 23.72ms-unsure 🔍
-2% - +2%
-0.57ms - +0.44ms		unsure 🔍
-2% - +3%
-0.43ms - +0.60ms
preact-local23.08ms - 23.66msunsure 🔍
-2% - +2%
-0.44ms - +0.57ms		-unsure 🔍
-1% - +2%
-0.29ms - +0.57ms
preact-hooks22.91ms - 23.54msunsure 🔍
-3% - +2%
-0.60ms - +0.43ms		unsure 🔍
-2% - +1%
-0.57ms - +0.29ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master1.93ms - 1.93ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms		faster ✔
1% - 1%
0.02ms - 0.02ms
preact-local1.93ms - 1.93msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-faster ✔
1% - 1%
0.02ms - 0.02ms
preact-hooks1.95ms - 1.95msslower ❌
1% - 1%
0.02ms - 0.02ms		slower ❌
1% - 1%
0.02ms - 0.02ms		-
hydrate1k

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master124.52ms - 126.68ms-unsure 🔍
-2% - +0%
-2.45ms - +0.54ms		unsure 🔍
-1% - +1%
-1.70ms - +1.11ms
preact-local125.52ms - 127.59msunsure 🔍
-0% - +2%
-0.54ms - +2.45ms		-unsure 🔍
-1% - +2%
-0.72ms - +2.03ms
preact-hooks125.00ms - 126.80msunsure 🔍
-1% - +1%
-1.11ms - +1.70ms		unsure 🔍
-2% - +1%
-2.03ms - +0.72ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master14.80ms - 15.60ms-unsure 🔍
-4% - +4%
-0.61ms - +0.53ms		unsure 🔍
-3% - +5%
-0.39ms - +0.70ms
preact-local14.83ms - 15.64msunsure 🔍
-4% - +4%
-0.53ms - +0.61ms		-unsure 🔍
-2% - +5%
-0.36ms - +0.74ms
preact-hooks14.67ms - 15.41msunsure 🔍
-5% - +3%
-0.70ms - +0.39ms		unsure 🔍
-5% - +2%
-0.74ms - +0.36ms		-
many_updates

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master35.11ms - 36.44ms-unsure 🔍
-4% - +2%
-1.38ms - +0.61ms		unsure 🔍
-5% - +0%
-1.72ms - +0.19ms
preact-local35.42ms - 36.89msunsure 🔍
-2% - +4%
-0.61ms - +1.38ms		-unsure 🔍
-4% - +2%
-1.39ms - +0.62ms
preact-hooks35.85ms - 37.22msunsure 🔍
-1% - +5%
-0.19ms - +1.72ms		unsure 🔍
-2% - +4%
-0.62ms - +1.39ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master4.97ms - 4.97ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms		unsure 🔍
-0% - -0%
-0.02ms - -0.02ms
preact-local4.97ms - 4.97msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-unsure 🔍
-0% - -0%
-0.02ms - -0.02ms
preact-hooks4.99ms - 4.99msunsure 🔍
+0% - +0%
+0.02ms - +0.02ms		unsure 🔍
+0% - +0%
+0.02ms - +0.02ms		-
text_update

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master2.98ms - 3.04ms-unsure 🔍
-2% - +1%
-0.05ms - +0.04ms		faster ✔
4% - 7%
0.14ms - 0.23ms
preact-local2.98ms - 3.04msunsure 🔍
-1% - +2%
-0.04ms - +0.05ms		-faster ✔
4% - 7%
0.13ms - 0.23ms
preact-hooks3.15ms - 3.23msslower ❌
4% - 8%
0.14ms - 0.23ms		slower ❌
4% - 8%
0.13ms - 0.23ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master1.14ms - 1.14ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms		faster ✔
1% - 1%
0.01ms - 0.01ms
preact-local1.14ms - 1.14msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-faster ✔
1% - 1%
0.01ms - 0.01ms
preact-hooks1.16ms - 1.16msslower ❌
1% - 1%
0.01ms - 0.01ms		slower ❌
1% - 1%
0.01ms - 0.01ms		-
todo

duration

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master68.38ms - 71.88ms-unsure 🔍
-5% - +2%
-3.42ms - +1.57ms		unsure 🔍
-3% - +3%
-2.16ms - +2.02ms
preact-local69.27ms - 72.83msunsure 🔍
-2% - +5%
-1.57ms - +3.42ms		-unsure 🔍
-2% - +4%
-1.27ms - +2.97ms
preact-hooks69.05ms - 71.35msunsure 🔍
-3% - +3%
-2.02ms - +2.16ms		unsure 🔍
-4% - +2%
-2.97ms - +1.27ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-localvs preact-hooks
preact-master1.43ms - 1.43ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms		faster ✔
2% - 2%
0.03ms - 0.03ms
preact-local1.43ms - 1.43msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-faster ✔
2% - 2%
0.03ms - 0.03ms
preact-hooks1.46ms - 1.46msslower ❌
2% - 2%
0.03ms - 0.03ms		slower ❌
2% - 2%
0.03ms - 0.03ms		-

tachometer-reporter-action v2 for Benchmarks

@coveralls
Copy link

coveralls commented Oct 27, 2022 •
edited

Coverage Status

Coverage remained the same at 99.531% when pulling 443dcd1 on run-bench-workflow into 05d91ad on master.

@github-actions
Copy link

Size Change: 0 B

Total Size: 52.9 kB

ℹ️ View Unchanged
Filename Size Change
compat/dist/compat.js 3.81 kB 0 B
compat/dist/compat.module.js 3.75 kB 0 B
compat/dist/compat.umd.js 3.87 kB 0 B
debug/dist/debug.js 3.01 kB 0 B
debug/dist/debug.module.js 3.01 kB 0 B
debug/dist/debug.umd.js 3.09 kB 0 B
devtools/dist/devtools.js 231 B 0 B
devtools/dist/devtools.module.js 240 B 0 B
devtools/dist/devtools.umd.js 314 B 0 B
dist/preact.js 4.06 kB 0 B
dist/preact.min.js 4.08 kB 0 B
dist/preact.min.module.js 4.08 kB 0 B
dist/preact.min.umd.js 4.11 kB 0 B
dist/preact.module.js 4.07 kB 0 B
dist/preact.umd.js 4.12 kB 0 B
hooks/dist/hooks.js 1.49 kB 0 B
hooks/dist/hooks.module.js 1.52 kB 0 B
hooks/dist/hooks.umd.js 1.57 kB 0 B
jsx-runtime/dist/jsxRuntime.js 358 B 0 B
jsx-runtime/dist/jsxRuntime.module.js 324 B 0 B
jsx-runtime/dist/jsxRuntime.umd.js 439 B 0 B
test-utils/dist/testUtils.js 442 B 0 B
test-utils/dist/testUtils.module.js 444 B 0 B
test-utils/dist/testUtils.umd.js 526 B 0 B

compressed-size-action

@andrewiggins
Copy link
Member Author

Manually ran benchmarks earlier and they worked successfully: https://github.com/preactjs/preact/actions/runs/3341614285

@andrewiggins andrewiggins merged commit 2a3b1aa into master Oct 28, 2022
@andrewiggins andrewiggins deleted the run-bench-workflow branch October 28, 2022 05:38
@marvinhagemeister marvinhagemeister mentioned this pull request Nov 14, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marvinhagemeister marvinhagemeister marvinhagemeister approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@andrewiggins @coveralls @marvinhagemeister