Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use artifact flow to report benchmarks results #2953

Merged
merged 2 commits into from Jan 23, 2021
Merged

Use artifact flow to report benchmarks results #2953

merged 2 commits into from Jan 23, 2021

Conversation

andrewiggins
Copy link
Member

@andrewiggins andrewiggins commented Jan 22, 2021 •
edited

Instead of reporting results in each benchmark job (which requires coordinating updating the comment between the concurrent jobs), we'll collect all the results in an artifact and then report the results after the benchmarks have finished.

This change dramatically reduces the number GitHub APIs calls and so should reduce the likelihood of our action getting throttled. The action still does more API calls than I like but figured I get this update out first to reduce the current strain.

Further, this serves as a building block to setting up the flow to enable benchmark reports for PRs from forks!! :D

@coveralls
Copy link

coveralls commented Jan 23, 2021 •
edited

Coverage Status

Coverage remained the same at 99.44% when pulling 10178e0 on tachometer-reporter-artifact-flow into cbe3369 on master.

@github-actions
Copy link

github-actions bot commented Jan 23, 2021 •
edited

Size Change: 0 B

Total Size: 42.1 kB

ℹ️ View Unchanged
Filename Size Change
compat/dist/compat.js 3.45 kB 0 B
compat/dist/compat.module.js 3.47 kB 0 B
compat/dist/compat.umd.js 3.51 kB 0 B
debug/dist/debug.js 2.99 kB 0 B
debug/dist/debug.module.js 2.98 kB 0 B
debug/dist/debug.umd.js 3.07 kB 0 B
devtools/dist/devtools.js 232 B 0 B
devtools/dist/devtools.module.js 240 B 0 B
devtools/dist/devtools.umd.js 308 B 0 B
dist/preact.js 3.93 kB 0 B
dist/preact.min.js 3.96 kB 0 B
dist/preact.module.js 3.96 kB 0 B
dist/preact.umd.js 4 kB 0 B
hooks/dist/hooks.js 1.13 kB 0 B
hooks/dist/hooks.module.js 1.15 kB 0 B
hooks/dist/hooks.umd.js 1.2 kB 0 B
jsx-runtime/dist/jsxRuntime.js 327 B 0 B
jsx-runtime/dist/jsxRuntime.module.js 335 B 0 B
jsx-runtime/dist/jsxRuntime.umd.js 405 B 0 B
test-utils/dist/testUtils.js 437 B 0 B
test-utils/dist/testUtils.module.js 439 B 0 B
test-utils/dist/testUtils.umd.js 515 B 0 B

compressed-size-action

@github-actions
Copy link

github-actions bot commented Jan 23, 2021 •
edited

📊 Tachometer Benchmark Results

Summary

duration

  • 02_replace1k: unsure 🔍 -3% - +1% (-5.84ms - +1.10ms)
    preact-local vs preact-master
  • 03_update10th1k_x16: unsure 🔍 -6% - +2% (-1.78ms - +0.47ms)
    preact-local vs preact-master
  • 07_create10k: unsure 🔍 -0% - +0% (-5.46ms - +4.87ms)
    preact-local vs preact-master
  • filter_list: unsure 🔍 -2% - +4% (-0.83ms - +1.56ms)
    preact-local vs preact-master
  • hydrate1k: unsure 🔍 -2% - +2% (-2.02ms - +1.86ms)
    preact-local vs preact-master
  • many_updates: unsure 🔍 -8% - +6% (-2.59ms - +2.15ms)
    preact-local vs preact-master
  • text_update: unsure 🔍 -5% - +5% (-0.17ms - +0.17ms)
    preact-local vs preact-master

usedJSHeapSize

  • 02_replace1k: unsure 🔍 -0% - +0% (-0.01ms - +0.00ms)
    preact-local vs preact-master
  • 03_update10th1k_x16: unsure 🔍 -0% - +0% (-0.01ms - +0.00ms)
    preact-local vs preact-master
  • 07_create10k: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • filter_list: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • hydrate1k: unsure 🔍 -0% - +0% (-0.00ms - +0.02ms)
    preact-local vs preact-master
  • many_updates: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master
  • text_update: unsure 🔍 -0% - +0% (-0.00ms - +0.00ms)
    preact-local vs preact-master

Results

02_replace1k

duration

VersionAvg timevs preact-mastervs preact-local
preact-master185.03ms - 189.83ms-unsure 🔍
-1% - +3%
-1.10ms - +5.84ms
preact-local182.56ms - 187.56msunsure 🔍
-3% - +1%
-5.84ms - +1.10ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master3.59ms - 3.60ms-unsure 🔍
-0% - +0%
-0.00ms - +0.01ms
preact-local3.59ms - 3.60msunsure 🔍
-0% - +0%
-0.01ms - +0.00ms		-

run-warmup-0

VersionAvg timevs preact-mastervs preact-local
preact-master70.42ms - 72.68ms-unsure 🔍
-1% - +3%
-0.86ms - +2.20ms
preact-local69.84ms - 71.91msunsure 🔍
-3% - +1%
-2.20ms - +0.86ms		-

run-warmup-1

VersionAvg timevs preact-mastervs preact-local
preact-master105.95ms - 108.74ms-unsure 🔍
-2% - +2%
-1.89ms - +2.47ms
preact-local105.39ms - 108.73msunsure 🔍
-2% - +2%
-2.47ms - +1.89ms		-

run-warmup-2

VersionAvg timevs preact-mastervs preact-local
preact-master121.63ms - 126.15ms-unsure 🔍
-2% - +2%
-3.10ms - +1.98ms
preact-local123.28ms - 125.61msunsure 🔍
-2% - +3%
-1.98ms - +3.10ms		-

run-warmup-3

VersionAvg timevs preact-mastervs preact-local
preact-master70.38ms - 73.75ms-unsure 🔍
-4% - +2%
-2.64ms - +1.69ms
preact-local71.19ms - 73.90msunsure 🔍
-2% - +4%
-1.69ms - +2.64ms		-

run-warmup-4

VersionAvg timevs preact-mastervs preact-local
preact-master135.65ms - 141.72ms-unsure 🔍
-4% - +1%
-6.15ms - +1.96ms
preact-local138.09ms - 143.47msunsure 🔍
-1% - +4%
-1.96ms - +6.15ms		-

run-final

VersionAvg timevs preact-mastervs preact-local
preact-master71.23ms - 73.62ms-unsure 🔍
-0% - +4%
-0.26ms - +3.02ms
preact-local69.92ms - 72.17msunsure 🔍
-4% - +0%
-3.02ms - +0.26ms		-
03_update10th1k_x16

duration

VersionAvg timevs preact-mastervs preact-local
preact-master29.26ms - 30.88ms-unsure 🔍
-2% - +6%
-0.47ms - +1.78ms
preact-local28.64ms - 30.20msunsure 🔍
-6% - +2%
-1.78ms - +0.47ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master3.52ms - 3.52ms-unsure 🔍
-0% - +0%
-0.00ms - +0.01ms
preact-local3.51ms - 3.52msunsure 🔍
-0% - +0%
-0.01ms - +0.00ms		-
07_create10k

duration

VersionAvg timevs preact-mastervs preact-local
preact-master1449.49ms - 1455.95ms-unsure 🔍
-0% - +0%
-4.87ms - +5.46ms
preact-local1448.40ms - 1456.46msunsure 🔍
-0% - +0%
-5.46ms - +4.87ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master25.98ms - 25.99ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms
preact-local25.98ms - 25.99msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-
filter_list

duration

VersionAvg timevs preact-mastervs preact-local
preact-master34.64ms - 36.23ms-unsure 🔍
-4% - +2%
-1.56ms - +0.83ms
preact-local34.90ms - 36.70msunsure 🔍
-2% - +4%
-0.83ms - +1.56ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master1.60ms - 1.60ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms
preact-local1.60ms - 1.60msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-
hydrate1k

duration

VersionAvg timevs preact-mastervs preact-local
preact-master115.70ms - 118.27ms-unsure 🔍
-2% - +2%
-1.86ms - +2.02ms
preact-local115.46ms - 118.36msunsure 🔍
-2% - +2%
-2.02ms - +1.86ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master6.19ms - 6.19ms-unsure 🔍
-0% - +0%
-0.02ms - +0.00ms
preact-local6.19ms - 6.21msunsure 🔍
-0% - +0%
-0.00ms - +0.02ms		-
many_updates

duration

VersionAvg timevs preact-mastervs preact-local
preact-master31.90ms - 35.69ms-unsure 🔍
-6% - +8%
-2.15ms - +2.59ms
preact-local32.15ms - 35.00msunsure 🔍
-8% - +6%
-2.59ms - +2.15ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master4.84ms - 4.84ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms
preact-local4.84ms - 4.84msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-
text_update

duration

VersionAvg timevs preact-mastervs preact-local
preact-master3.57ms - 3.78ms-unsure 🔍
-5% - +5%
-0.17ms - +0.17ms
preact-local3.54ms - 3.81msunsure 🔍
-5% - +5%
-0.17ms - +0.17ms		-

usedJSHeapSize

VersionAvg timevs preact-mastervs preact-local
preact-master0.83ms - 0.83ms-unsure 🔍
-0% - +0%
-0.00ms - +0.00ms
preact-local0.83ms - 0.83msunsure 🔍
-0% - +0%
-0.00ms - +0.00ms		-

tachometer-reporter-action v2 for Benchmarks

@andrewiggins andrewiggins force-pushed the tachometer-reporter-artifact-flow branch 2 times, most recently from 3f17014 to 672f4dc Compare January 23, 2021 02:35
@andrewiggins andrewiggins force-pushed the tachometer-reporter-artifact-flow branch 2 times, most recently from b6d21d5 to 19b8fca Compare January 23, 2021 02:59
@andrewiggins andrewiggins marked this pull request as ready for review January 23, 2021 17:55
@andrewiggins andrewiggins merged commit 359ad4f into master Jan 23, 2021
@andrewiggins andrewiggins deleted the tachometer-reporter-artifact-flow branch January 23, 2021 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marvinhagemeister marvinhagemeister marvinhagemeister approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@andrewiggins @coveralls @marvinhagemeister