Boost.CI

This repository contains scripts that enable continuous integration (CI) with Appveyor, Azure Pipelines, codecov.io, Coverity Scan, GitHub Actions, Drone, and Travis CI. These scripts are intended to be downloaded and used during boost repository builds to improve project quality. In most cases the scripts are self-configuring. Some integrations require additional setup actions to complete.

Boost.CI also allows you to run a big-endian build on Travis CI and Github Actions.

Build Status

GH Actions	Appveyor	Azure Pipelines	Drone	codecov.io

Summary (TL;DR)

Here are all the steps you need to take as a Boost repository maintainer to enable all of these CI features in your repository. (Note that you may skip some steps, e.g. if you don't need a specific CI service):

1. Checkout develop and then make a new branch called ci.
2. Copy the .appveyor.yml file from this repository into the top level of your repository.
3. Copy the .azure-pipelines.yml file from this repository into the top level of your repository.
4. Copy the .travis.yml file from this repository into the top level of your repository.
5. Copy the .github/workflows/ci.yml file from this repository into the the same folder in your repository.
   • Note that those include CMake tests, so see the CMake docs for details and requirements.
6. Copy the .drone.star file and optionally the .drone directory from this repository to the top level of your repository.
7. Copy the .codecov.yml file from this repository to the top level of your repository and edit if required. Note that only the file in the default branch (usually master) is considered!
8. Copy the LICENSE file from this repository to the top level of your repository. This adds the BSL-1.0 designation to your repository on github.
9. [optional] Copy the README.template.md file from this repository to the top level README.md of your repository. If you already have a README.md then you can take what you need from the template version to improve it, if desired. Otherwise, you will need to customize README.md for your repository. One useful step is to fixup the repository name using the command sed -i 's/template/<myrepositoryname>/g' README.md, and then update the first line description.
10. In Appveyor, add a project for your fork of the repository. No customization is needed.
11. In Travis CI, add a project for your fork of the repository. Later you will customize it for Coverity Scan, but for now no settings changes are necessary.
12. Commit these changes and push to your personal fork in the ci branch.
13. Create a pull request in your fork of /ci to /develop. Do not target boostorg/develop.
14. Observe that the CI services are running the build jobs. Fix up any issues found. Note this may uncover defects in your repository code.
15. If you are the owner or an admin for your repository, add projects in Appveyor and Travis CI for the boostorg/ project (not your fork). If you are just a contributor in the repository, create an issue in Boost.Admin requesting Appveyor and Travis CI to be enabled for the repository.
16. Commit the changes to develop. This will kick off a build.
17. Update the badge matrix in README.md with the correct links for your CI projects in use (e.g. Appveyor, Github Actions, Travis).
18. Create a Coverity Scan account if you have not already done so.
19. Create a new Coverity Scan github based project for your official boostorg repository.
20. Update your Travis CI and/or Github Actions boostorg repository project settings and add the following environment variables using respective CI GUI:
    • COVERITY_SCAN_NOTIFICATION_EMAIL can be public and set to your email account (or it can be private).
    • COVERITY_SCAN_TOKEN should be kept private (aka. "secret") and set to the scan token you can find in the project settings in Coverity Scan.
21. Update the README.md to put the correct Coverity Scan badge project number into the badge URLs.
22. This will kick off a build on the develop branch that will include Coverity Scan results.
23. To activate Drone, visit https://drone.cpp.al. Authorize Drone: Click the "Authorize cppalliance-drone" button. Sync repositories: Click the "sync" button. A list of repositories will appear. For the relevant repo, click and then choose "Activate Repository". In the settings page, change Configuration from .drone.yml to .drone.star. "Save".
24. More pointers about Drone:
    • Ensure that shell scripts are executable: chmod 755 .drone/drone.sh
    • The install-script (in ci/drone) and run-script (in .drone) for the Unix-ish jobs will be downloaded from Boost.CI if they don't exist, so you only need them when you want to customize the build.
    • The .drone/{before,after}-install.* scripts are sourced around the common_install step (which e.g. bootstraps B2) of the default build, if they exist. So you can remove them when not required.
    • "asan" jobs require elevated privileges. Contact an administrator or open an issue at drone-ci to set your drone repository to "Trusted".
    • If not using asan, simply remove the jobs.
    • For Codecov you need to copy the "Repository Upload Token" from the settings page of your repo on Codecov and use it to create a new secret named codecov_token on the settings page of your repo on Drone.
    • If you need a package installed on MacOS or FreeBSD, by the root user, please open an issue.
    • Further info available at https://github.com/CPPAlliance/drone-ci

Code coverage

Multiple CI configs contain jobs collecting code coverage data. E.g. Github Actions and Drone CI for Linux and Appveyor for Windows. Especially the latter allows to collect coverage data for Windows-only codeparts or code using e.g. wchar_t which is different on Windows than on other platforms.

Exclusion of coverage data

If you want to exclude parts of your code from coverage analysis you can use the LCOV comments/macros:

• // LCOV_EXCL_LINE for a single line
• // LCOV_EXCL_START and // LCOV_EXCL_STOP for a range of code

See the LCov manual for more information.

To exclude whole files or folders you can use the ignore-object and glob patterns in in the .codecov.yml file. See the example file in the root of this repository for a starting point.
Important: Codecov only considers the configuration file of the default branch (usually: master), not the one in the current build/PR branch. See the CodeCov documentation for details.

Repositories using Boost.CI

The CMT Stale Repo Tracker identifies many repositories using Boost.CI and the CMT Status Spreadsheet shows the current state of each. There may be additional repositories using Boost.CI that are not listed. Boost.CI does not track usage internally.

How It Works

The CI config files (such as .appveyor.yml, .azure-pipeline.yml and .github/workflows/ci.yml) must exist in your repository and will contain your customizations for build types, languages, and platforms. The templates provided will get you started with the build jobs listed below.

These scripts will copy resources from the Boost.CI repository when needed in order to provide scripting necessary to run all these jobs successfully.

Build jobs that will severely impact performance (such as valgrind) will define BOOST_NO_STRESS_TEST so those can be skipped or hobbled.

Topic Branch Support

The configuration for Github Actions, Azure Pipelines, Appveyor and Travis CI allow for automated branch builds on branch pushes matching these names:

• master
• develop
• bugfix/*
• feature/*
• fix/*
• pr/*

Note that when opening a Pull Request (PR) the use of a topic branch name is not required and even discouraged as PRs are always built on CI and the use of a topic branch name will lead to duplicating all builds.

A good strategy is to only use a topic branch name when actively working on a changeset and when you want CI to run for each push of that branch. When the changeset is (mostly) done it is advised to rename the branch right before opening a PR to avoid that double-build issue. This can be done easily via the GitHub website (check the "branches" link at the repo overview page) or of course via command line (push to a new remote branch and delete the old one).

Defaults, Builds and Services

By default the builds target multiple different C++ versions (from C++03 to C++20), and this can be customized. To see what kind of coverage these builds provide, see some build results as follows or click the badges above:

AppVeyor : https://ci.appveyor.com/project/Flamefire/boost-ci/branch/master
Github Actions : https://github.com/boostorg/boost-ci/actions/workflows/ci.yml
Azure Pipelines : https://dev.azure.com/boostorg/boost-ci/_build/latest?definitionId=8&branchName=master
Travis CI : https://travis-ci.org/boostorg/uuid/builds/449557162

Without any customization the scripts can provide the following services (example only, see the actual CI scripts for current configurations):

CI	description	toolset	cxxflags/std	address-model	variant
Appveyor	MSVC 2019 C++2a Strict	msvc-14.2	2a, -permissive-	64	release
Appveyor	MSVC 2017 C++2a Strict	msvc-14.1	2a, -permissive-	64	release
Appveyor	MSVC 2017 C++17	msvc-14.1	17	64	debug
Appveyor	MSVC 2017 C++17	clang-win	11	64	release
Appveyor	MSVC 2017 C++14 Default	msvc-14.1	default (14)	32,64	release
Appveyor	MSVC 2015 C++14 Default	msvc-14.0	default (14)	32,64	debug
Appveyor	MSVC 2013	msvc-12.0	default (most of 11)	default	release
Appveyor	MSVC 2012	msvc-11.0	default (some of 11)	default	release
Appveyor	MSVC 2010	msvc-10.0	default (some of 0x)	default	release
Appveyor	cygwin	gcc	03,11	32	debug
Appveyor	cygwin64	gcc	11,17	64	release
Appveyor	mingw	gcc	03,11	32	debug
Appveyor	mingw64	gcc	11,17	64	release
Azure P.	gcc 4.8	gcc-4.8	03,11	default	debug,release
Azure P.	gcc 4.9	gcc-4.9	03,11	default	debug,release
Azure P.	gcc 5	gcc-5	11	default	debug,release
Azure P.	gcc 6	gcc-6	11,14	default	debug,release
Azure P.	gcc 7	gcc-7	11,14,17	default	debug,release
Azure P.	gcc 8	gcc-8	14,17,2a	default	debug,release
Azure P.	clang-3.5	clang-3.5	03,11	default	debug,release
Azure P.	clang-3.6	clang-3.6	03,11	default	debug,release
Azure P.	clang-3.7	clang-3.7	03,11	default	debug,release
Azure P.	clang-3.8	clang-3.8	03,11,14	default	debug,release
Azure P.	clang-3.9	clang-3.8	03,11,14	default	debug,release
Azure P.	clang-4.0	clang-4.0	11,14,17	default	debug,release
Azure P.	clang-5.0	clang-5.0	11,14,17	default	debug,release
Azure P.	clang-6.0	clang-6.0	14,17,2a	default	debug,release
Azure P.	clang-6.0-libc++	clang-6.0	03,11,14,17,2a, libc++	default	debug,release
Azure P.	clang-7	clang-7	14,17,2a	default	debug,release
Azure P.	clang-8	clang-8	14,17,2a	default	debug,release
Azure P.	MSVC 2019 C++2a Strict	msvc-14.2	2a, -permissive-	64	debug,release
Azure P.	MSVC 2017 C++2a Strict	msvc-14.1	2a, -permissive-	64	debug,release
Azure P.	MSVC 2017 C++17	msvc-14.1	17	32,64	debug,release
Azure P.	MSVC 2017 C++14 Default	msvc-14.1	default (14)	32,64	debug,release
Azure P.	MSVC 2015 C++14 Default	msvc-14.0	default (14)	32,64	debug,release
Azure P.	Xcode 10.1	clang	14,17,2a	default	debug,release
Azure P.	Xcode 10.0	clang	14,17,2a	default	debug,release
Azure P.	Xcode 9.4.1	clang	11,14,17	default	debug,release
Azure P.	Xcode 9.4	clang	11,14,17	default	debug,release
Azure P.	Xcode 9.3.1	clang	11,14	default	debug,release
Azure P.	Xcode 9.3	clang	11,14	default	debug,release
Azure P.	Xcode 9.2	clang	11,14	default	debug,release
Azure P.	Xcode 9.1	clang	03,11	default	debug,release
Azure P.	Xcode 9.0.1	clang	03,11	default	debug,release
Azure P.	Xcode 9.0	clang	03,11	default	debug,release
Azure P.	Xcode 8.3.3	clang	03,11	default	debug,release
Travis CI	gcc 4.8	gcc-4.8	03,11	default	release
Travis CI	gcc 4.9	gcc-4.9	03,11	default	release
Travis CI	gcc 5	gcc-5	03,11	default	release
Travis CI	gcc 6	gcc-6	11,14	default	release
Travis CI	gcc 7	gcc-7	14,17	default	release
Travis CI	gcc 8	gcc-8	17,2a	default	release
Travis CI	gcc 9	gcc-9	17,2a	default	release
Travis CI	clang-3.8	clang-3.8	03,11	default	release
Travis CI	clang-4.0	clang-4.0	11,14	default	release
Travis CI	clang-5.0	clang-5.0	11,14	default	release
Travis CI	clang-6.0	clang-6.0	14,17	default	release
Travis CI	clang-6.0-libc++	clang-6.0	03,11,14, libc++	default	release
Travis CI	clang-7	clang-7	17,2a	default	release
Travis CI	clang-8	clang-8	17,2a	default	release
Travis CI	osx (clang)	clang	03,11,17	default	release
Travis CI	big-endian	gcc	default	default	debug
Travis CI	codecov.io	gcc-8	default	default	debug
Travis CI	covscan	clang	default	default	debug
Travis CI	asan	gcc-8	03,11,14	default	debug
Travis CI	tsan	gcc-8	03,11,14	default	debug
Travis CI	ubsan	gcc-8	03,11,14	default	debug
Travis CI	valgrind	clang-6.0	03,11,14	default	debug