CI with Packit in GitHub CI using Fedora COPR as build service

[bernhardkaindl]

Hi @namhyung!

This PR adds the configuration to support using Fedora Packit for GitHub CI using Fedora COPR.

---

As a demonstration, please have a look at the Checks executed for this PR here:

In linux-tracing#3, to the right of:

All checks have passed
12 successful checks

-> click on the link "show all checks" to see all the checks, and scroll down the list of checks (of course it is configurable)

---

This PR provides this capability, when Fedora Packit (which is amed to help devs) is enabled for a given Repo,
it does nothing when Packit is not enabled.

What you get:

• Uses Fedora COPR, the free build service of Fedora builds RPMs for CI testing
• The testsuite is executed during each RPM build (it allows to put this into a separate step as well)
• The Output is checked and archived in the RPM package for analysis.
  • The output of the build is accessible directly from the GitHub Checks.
  • All output is preseved for 60 days for inspection, including testsuite logs.
• If the testsuite shows regressions, the build fails.
  • For now, the limits which cause the build to fails are defined in the spec file
  • Of course, this could best be moved into a script or test/run_testsuite.py
• For each PR, it builds with Fedora 36, 37, 38 and 39 (x86_64 and aarch64)
  • Because the builds happen on the latest Fedora code, you get early warning if something breaks.
• The duration all builds and full testsuite runs should be roughly within 20 minutes on average.

This page gives you a quick overview how it works: https://packit.dev (Quotes from it):

What is Packit?

An open source project aiming to ease the integration of your project with Fedora Linux, CentOS Stream and other distributions.

Who should use Packit?

Upstream projects that want to build RPM packages out of pull requests, commits or releases, test their code changes and bring upstream releases to Fedora Linux.

This describes how to enable the Packit Service: https://packit.dev/docs/guide/, in short:

1. An account at https://accounts.fedoraproject.org/ (you can just sign up) needs set your user name as his Github user.

2. The administrator of the repo (likely you), would need to "order" the free Packit-as-a-service on Github.

3. Then, using a special comment on a Github PR, you confirm that the user on accounts.fedoraproject.org should be used to run the builds,

4. Create a pull request for master and have fun ;-)

The used uftrace.spec unfortunately excludes a number of tests from the testsuite runs because they tended to fail in most build environments, but it keeps most tests and defines upper bounds for the number of fails, classified as differing, failing to build, non-zero exit status, etc. This reduction of number of fails can of course be omitted, and the numbers of tolerated fails would have to be raised to make the builds still succeed (or when figured out, moved into a separate check stage, but running all in one go has the benefit that in this way, the test reports are included in the RPM packages and can analysed and compared from there.)

PS: Currently, some test suites fail differently in each build environment, so the limits are oriented at those build environments with the highest amount of failures and skipped tests in order to make the builds pass under normal circumstances.

But the good thing is that with this regressions can be noticed earlier (some seem to be between 0.13 and master):

This CI addition ensures that each merge into uftrace is also buildable for Fedora (CentOS Stream can be added too) and has (at least in the tests which are not excluded or skipped) if at all only a limited amount of regressions in any of the defined RPM build environments for these distributions.

[namhyung]

Nice work! I'm happy to see CI working on Fedora.

[namhyung]

It'd be nice if I can trigger it when I push some codes to a branch. For example, I often push some code to the branches under check/ or review/. I usually don't create a PR by myself.

Also I want to run different set of tests depending on the trigger type. For PR check, maybe it's enough run the tests on the latest Fedora only. For nightly test, it can run on most versions.

[bernhardkaindl]

Yes, it's possible to trigger builds by commits to specific branches:
https://packit.dev/docs/configuration/#packit-service-jobs
Example snippet for .packit.yml from this page:

- job: copr_build
  trigger: commit
  branch: main
  targets:
  - centos-stream-8-x86_64
  - centos-stream-9-x86_64
  - fedora-all

So yes, depending on the branch, you can configure specific targets to be built.

I don't know or understand yet, where the builds will be linked so you can access them, but I assume that it just has to be tried, and then we'll find out.

[namhyung]

Is it also possible to trigger the build manually like from nightly tests?

[bernhardkaindl]

There seems to be a possiblitily run jobs on a daily basis. At least, there is a project for daily LLVM snapshot builds:
https://copr.fedorainfracloud.org/coprs/g/fedora-llvm-team/llvm-snapshots/

A likewise also interesting topic is the Fedora testing farm which can be also used thruogh Packit:
https://fedoramagazine.org/test-github-projects-with-github-actions-and-testing-farm/

For public users, the tests are run only by AWS instances, but it has a nice test results viewer:
https://docs.testing-farm.io/general/0.1/test-results.html

[namhyung]

Can these files live in the misc/ directory?