chore: update lerna since parameter for push github action

[chigia001]

Which problem is this PR solving?

Fixes #1473

• GH's action is setup with --since origin/main. This is correct for PR but not for commit that push on main since this will alway return empty change set.

Short description of the changes

• move since argument to unit-test.yml and base commit base on event

[codecov]

Codecov Report

Merging #1580 (d78dc7c) into main (b5fc0c4) will decrease coverage by 4.29%.
The diff coverage is 92.77%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1580      +/-   ##
==========================================
- Coverage   96.06%   91.77%   -4.29%     
==========================================
  Files          14      137     +123     
  Lines         914     7064    +6150     
  Branches      199     1424    +1225     
==========================================
+ Hits          878     6483    +5605     
- Misses         36      581     +545     

Impacted Files	Coverage Δ
...de/instrumentation-cucumber/src/instrumentation.ts	92.30% <92.30%> (ø)
...tapackages/auto-instrumentations-node/src/utils.ts	98.78% <100.00%> (ø)
plugins/node/instrumentation-cucumber/src/types.ts	100.00% <100.00%> (ø)

... and 120 files with indirect coverage changes

[chigia001]

@pichlermarc to fix #1473, for main branch should we run unit test for all package or just the change package introduce by new commit?

[blumamir]

I never fully understood how the code coverage works in details, but if I understand correctly, the goal is for pushes to main to always generate full report with all repo tests in codecov, where as now only web is reported:

In this case, wouldn't running the tests with --since github.event.push.before will cause partial reports that only tracks the changed package in each merge to main, but will skip packages that have not had any change in the merge?

I added this flag long ago in #614 and the goal was to speed up ci time for PRs, but I guess for merges to main it would be best to just run full tests without --since. I wonder what are your thoughts and if you support it - are aware of a way to configure it in the yml? We can also rethink if we want this optimization or just run all tests all the time (push to main and PRs). If I remember correctly, the"Unit test" CI step run time was few minutes, and not dramatic compared to overall workflow time. Since it confuses codecov, we can consider just removing the flag altogether.

[chigia001]

In this case, wouldn't running the tests with --since github.event.push.before will cause partial reports that only tracks the changed package in each merge to main, but will skip packages that have not had any change in the merge?

Yes, that also why I want to confirm this behavior with @pichlermarc

Codecov also support Flag feature with CarryForward, you can see one of the report here:
https://app.codecov.io/gh/open-telemetry/opentelemetry-js-contrib/commit/7b14c92d6913d104401ec086034168dfc57aedfe

In above commit, I configure to skip the node test but the codecov still capture the previous node's code coverage in single report.

But to fully support our codebase, each package must define it own flag/paths in codecov.yml which could become cumbersome.

If I remember correctly, the"Unit test" CI step run time was few minutes, and not dramatic compared to overall workflow time.

Yes, a full unit test is arround 5m, but it relative small compare to npm install + lerna bootstrap step which account for 30m of execution time.

[chigia001]

@blumamir I dig a little bit on to codecov's Components and Flags and think it could potential improve but would require more code change + some potential maintaince burden. I can create a prototype PR for that.

So I think for now it better to just do a full-run on node@14 to collect coverage report. For Node@16 and node@18 we can run a delta-run to speed thing up a bit, what do you think?

[blumamir]

Another thought. If I understand correctly, running codecov on main has 2 purposes:

1. to give an overview of the coverage for people browsing the project in codecov website
2. to act as a baseline for each PR commit, which codecov can use for coverage diff which generates the CI comment and make unit-test workflow either pass or fail.
   

Is it right?

In this case, should both the PR commit and main run the same set of tests so that the diff can be calculated accurately? Can this be achieved if we use the --since flag at all?

[chigia001]

@blumamir Codecov seem to have a carry forward flag for this use case
https://about.codecov.io/blog/new-product-test-only-what-you-change-with-carryforward-flags/

Let me test to see if this work

[chigia001]

closed in favor of #1605