Add Comments to PRs with Coverage Details

[mtwichel]

Is your feature request related to a problem? Please describe.
I really like the new feature where the lines missed will print in the output of the actions; however, I find it kinda annoying to go into the action details and click through to find what I am looking for.

Describe the solution you'd like
I think it would be really cool if the action could be optionally configured to write a comment on your PR with a brief report of your coverage stats. We could include if it passed, what percentage it calculated, and list any lines missed. When you re-ran the action, it would update the comment with the new details, so it wouldn't pollute the PR.

The feature is inspired by how the firebase hosting deploy action adds a comment with a preview URL. I think we can also take queues from their code for how to accomplish it.

I would propose adding new inputs as well:

• report_coverage_comment (boolean, defaults false): Set true if you'd like the action to comment on your PR
• project_name (string, optional): If set, prefixes the heading of the comment with the name (so that it's easy to identify which comment is for which project if you have multiple actions. Otherwise, defaults to the name of the workflow.

Describe alternatives you've considered
One alternative I've played with is adding outputs to the existing action - that way, subsequent steps could post the comment (like using this action). That works well enough (I have a fork working now), but it really requires a fair bit of work to make your workflow work (for example, conditionally writing a new comment vs update existing; formatting with markdown, ect). I'd rather have it contained in an action to make it easy to add to lots of projects.

Of course, I could make a separate action that takes outputs from VGC as inputs and writes the comments. I think that would be a viable option, but would suffer discoverability and would be a fair bit more work to set up.

Thanks as always VGV! I'm happy to help make this a reality, but I think there's some debate on the best way to set this up :)

[mtwichel]

Of course, I could make a separate action that takes outputs from VGC as inputs and writes the comments. I think that would be a viable option, but would suffer discoverability and would be a fair bit more work to set up.

If we go this route, on the step that calls the new action, we would have to add

if: ${{ always() }}

This is because if you're below the coverage threshold, VGC will error and cause the workflow to terminate. Adding this line to the step forces it to run anyways (but the workflow still fails).

Not a big deal, but maybe a bit annoying to remember every time?

[definitelyokay]

Great idea! We can also draw inspiration from the comment posted by this coverage action: https://github.com/zgosalvez/github-actions-report-lcov

[felangel]

Could use https://github.com/leancodepl/dart-problem-matcher for inspiration as well

[DFelten]

The option to display the information as a job summary would also be great.

[alestiago]

Could use codecov for inspiration as well