Benchmark tests workflow

[martinkuba]

Which problem is this PR solving?

This is a follow-up to adding performance benchmark tests (#4105). In order for benchmark tests to provide value, it is important to track changes in performance over time.

Short description of the changes

This adds a Github workflow that runs all benchmark tests and publishes results to Github pages. This is similar to what the Collector is already doing here.

The PR is in a draft mode until self-hosted runner is available to this repo (the process of adding a self-hosted runner is progress).

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

The workflow has been tested in a fork. Here is a page with charts that it has generated.

[codecov]

Codecov Report

Merging #4144 (fecd2b0) into main (52f428a) will decrease coverage by 0.21%.
The diff coverage is n/a.

❗ Current head fecd2b0 differs from pull request most recent head 8102aae. Consider uploading reports for the commit 8102aae to get more accurate results

@@            Coverage Diff             @@
##             main    #4144      +/-   ##
==========================================
- Coverage   92.50%   92.30%   -0.21%     
==========================================
  Files         275      329      +54     
  Lines        7369     9421    +2052     
  Branches     1540     1980     +440     
==========================================
+ Hits         6817     8696    +1879     
- Misses        552      725     +173     

see 78 files with indirect coverage changes

[martinkuba]

I have updated the URL path in Github pages, so that the test results will be accessible under
https://open-telemetry.github.io/opentelemetry-js/benchmarks

[pichlermarc]

Thanks for adding this 🙂

[martinkuba]

@pichlermarc Thanks for the review. There is potentially one more change that I am wondering about - should we run this on release or on every merge to main? With merge to main, it would be easier to catch regressions. What do you think?

[pichlermarc]

@pichlermarc Thanks for the review. There is potentially one more change that I am wondering about - should we run this on release or on every merge to main? With merge to main, it would be easier to catch regressions. What do you think?

👍 for running it on main 🙂

[dyladan]

We're already using gh-pages for the reference docs. Will this conflict with that?

[martinkuba]

We're already using gh-pages for the reference docs. Will this conflict with that?

No, it will not conflict. It's set to put the results in a benchmarks directory, so it will be available at https://open-telemetry.github.io/opentelemetry-js/benchmarks. This directory will contain only two files - HTML file and a JS data file. Here is an example.