Skip to content

Commit 9e1f8fc

Browse files
mmarchiniaddaleax
mmarchini
authored and
addaleax
committedSep 22, 2020
build: implement a Commit Queue in Actions
This is a (still experimental) implementation of a Commit Queue on GitHub Actions, using labels and the scheduler event to land Pull Requests. It uses `node-core-utils` to validate Pull Requests and to prepare the commit message, and then it uses a GitHub personal token to push changes back to the repository. If the Queue fails to land a Pull Request, that PR will be removed from the queue and the `node-core-utils` output will be pasted in the Pull Request. An overview of the implementation is provided in doc/guides/commit-queue.md, as well as current limitations. Ref: https://github.com/mmarchini-oss/automated-merge-test Ref: nodejs/build#2201 PR-URL: #34112 Refs: https://github.com/mmarchini-oss/automated-merge-test Refs: nodejs/build#2201 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Denys Otrishko <shishugi@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>
·
v12.22.12v12.19.0
1 parent 24fddba commit 9e1f8fc

File tree

4 files changed

+285
-6
lines changed

4 files changed

+285
-6
lines changed
 

‎.github/workflows/auto-start-ci.yml

+4-6
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,10 @@ name: Auto Start CI
33

44
on:
55
schedule:
6-
# `schedule` event is used instead of `pull_request` because when a
7-
# `pull_requesst` event is triggered on a PR from a fork, GITHUB_TOKEN will
8-
# be read-only, and the Action won't have access to any other repository
9-
# secrets, which it needs to access Jenkins API. Runs every five minutes
10-
# (fastest the scheduler can run). Five minutes is optimistic, it can take
11-
# longer to run.
6+
# Runs every five minutes (fastest the scheduler can run). Five minutes is
7+
# optimistic, it can take longer to run.
8+
# To understand why `schedule` is used instead of other events, refer to
9+
# ./doc/guides/commit-queue.md
1210
- cron: "*/5 * * * *"
1311

1412
jobs:

‎.github/workflows/commit-queue.yml

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
# This action requires the following secrets to be set on the repository:
3+
# GH_USER_NAME: GitHub user whose Jenkins and GitHub token are defined below
4+
# GH_USER_TOKEN: GitHub user token, to be used by ncu and to push changes
5+
# JENKINS_TOKEN: Jenkins token, to be used to check CI status
6+
7+
name: Commit Queue
8+
9+
on:
10+
# `schedule` event is used instead of `pull_request` because when a
11+
# `pull_request` event is triggered on a PR from a fork, GITHUB_TOKEN will
12+
# be read-only, and the Action won't have access to any other repository
13+
# secrets, which it needs to access Jenkins API.
14+
schedule:
15+
- cron: "*/5 * * * *"
16+
17+
jobs:
18+
commitQueue:
19+
if: github.repository == 'nodejs/node'
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@v2
23+
with:
24+
# A personal token is required because pushing with GITHUB_TOKEN will
25+
# prevent commits from running CI after they land on master. It needs
26+
# to be set here because `checkout` configures GitHub authentication
27+
# for push as well.
28+
token: ${{ secrets.GH_USER_TOKEN }}
29+
30+
# Install dependencies
31+
- name: Install Node.js
32+
uses: actions/setup-node@v2-beta
33+
with:
34+
node-version: '12'
35+
- name: Install dependencies
36+
run: |
37+
sudo apt-get install jq -y
38+
# TODO(mmarchini): install from npm after next ncu release is out
39+
npm install -g 'https://github.com/mmarchini/node-core-utils#commit-queue-branch'
40+
# npm install -g node-core-utils
41+
42+
- name: Set variables
43+
run: |
44+
echo "::set-env name=REPOSITORY::$(echo ${{ github.repository }} | cut -d/ -f2)"
45+
echo "::set-env name=OWNER::${{ github.repository_owner }}"
46+
47+
- name: Get Pull Requests
48+
uses: octokit/graphql-action@v2.x
49+
id: get_mergable_pull_requests
50+
with:
51+
query: |
52+
query release($owner:String!,$repo:String!, $base_ref:String!) {
53+
repository(owner:$owner, name:$repo) {
54+
pullRequests(baseRefName: $base_ref, labels: ["commit-queue"], states: OPEN, last: 100) {
55+
nodes {
56+
number
57+
}
58+
}
59+
}
60+
}
61+
owner: ${{ env.OWNER }}
62+
repo: ${{ env.REPOSITORY }}
63+
# Commit queue is only enabled for the default branch on the repository
64+
# TODO(mmarchini): get the default branch programmatically instead of
65+
# assuming `master`
66+
base_ref: "master"
67+
env:
68+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
69+
70+
- name: Configure node-core-utils
71+
run: |
72+
ncu-config set branch master
73+
ncu-config set upstream origin
74+
ncu-config set username "${{ secrets.GH_USER_NAME }}"
75+
ncu-config set token "${{ secrets.GH_USER_TOKEN }}"
76+
ncu-config set jenkins_token "${{ secrets.JENKINS_TOKEN }}"
77+
ncu-config set repo "${{ env.REPOSITORY }}"
78+
ncu-config set owner "${{ env.OWNER }}"
79+
80+
- name: Start the commit queue
81+
run: ./tools/actions/commit-queue.sh ${{ env.OWNER }} ${{ env.REPOSITORY }} ${{ secrets.GITHUB_TOKEN }} $(echo '${{ steps.get_mergable_pull_requests.outputs.data }}' | jq '.repository.pullRequests.nodes | map(.number) | .[]')

‎doc/guides/commit-queue.md

+115
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Commit Queue
2+
3+
> Stability: 1 - Experimental
4+
5+
*tl;dr: You can land Pull Requests by adding the `commit-queue` label to it.*
6+
7+
Commit Queue is an experimental feature for the project which simplifies the
8+
landing process by automating it via GitHub Actions. With it, Collaborators are
9+
able to land Pull Requests by adding the `commit-queue` label to a PR. All
10+
checks will run via node-core-utils, and if the Pull Request is ready to land,
11+
the Action will rebase it and push to master.
12+
13+
This document gives an overview on how the Commit Queue works, as well as
14+
implementation details, reasoning for design choices, and current limitations.
15+
16+
## Overview
17+
18+
From a high-level, the Commit Queue works as follow:
19+
20+
1. Collaborators will add `commit-queue` label to Pull Reuqests ready to land
21+
2. Every five minutes the queue will do the following for each Pull Request
22+
with the label:
23+
1. Check if the PR also has a `request-ci` label (if it has, skip this PR
24+
since it's pending a CI run)
25+
2. Check if the last Jenkins CI is finished running (if it is not, skip this
26+
PR)
27+
3. Remove the `commit-queue` label
28+
4. Run `git node land <pr>`
29+
5. If it fails:
30+
1. Abort `git node land` session
31+
2. Add `commit-queue-failed` label to the PR
32+
3. Leave a comment on the PR with the output from `git node land`
33+
4. Skip next steps, go to next PR in the queue
34+
6. If it succeeds:
35+
1. Push the changes to nodejs/node
36+
2. Leave a comment on the PR with `Landed in ...`
37+
3. Close the PR
38+
4. Go to next PR in the queue
39+
40+
## Current Limitations
41+
42+
The Commit Queue feature is still in early stages, and as such it might not
43+
work for more complex Pull Requests. These are the currently known limitations
44+
of the commit queue:
45+
46+
1. The Pull Request must have only one commit
47+
2. A CI must've ran and succeeded since the last change on the PR
48+
3. A Collaborator must have approved the PR since the last change
49+
4. Only Jenkins CI is checked (Actions, V8 CI and CITGM are ignored)
50+
51+
## Implementation
52+
53+
The [action](/.github/workflows/commit_queue.yml) will run on scheduler
54+
events every five minutes. Five minutes is the smallest number accepted by
55+
the scheduler. The scheduler is not guaranteed to run every five minutes, it
56+
might take longer between runs.
57+
58+
Using the scheduler is preferrable over using pull_request_target for two
59+
reasons:
60+
61+
1. if two Commit Queue Actions execution overlap, there's a high-risk that
62+
the last one to finish will fail because the local branch will be out of
63+
sync with the remote after the first Action pushes. `issue_comment` event
64+
has the same limitation.
65+
2. `pull_request_target` will only run if the Action exists on the base commit
66+
of a pull request, and it will run the Action version present on that
67+
commit, meaning we wouldn't be able to use it for already opened PRs
68+
without rebasing them first.
69+
70+
`node-core-utils` is configured with a personal token and
71+
a Jenkins token from
72+
[@nodejs-github-bot](https://github.com/nodejs/github-bot).
73+
`octokit/graphql-action` is used to fetch all Pull Requests with the
74+
`commit-queue` label. The output is a JSON payload, so `jq` is used to turn
75+
that into a list of PR ids we can pass as arguments to
76+
[`commit-queue.sh`](./tools/actions/commit-queue.sh).
77+
78+
> The personal token only needs permission for public repositories and to read
79+
> profiles, we can use the GITHUB_TOKEN for write operations. Jenkins token is
80+
> required to check CI status.
81+
82+
`commit-queue.sh` receives the following positional arguments:
83+
84+
1. The repository owner
85+
2. The repository name
86+
3. The Action GITHUB_TOKEN
87+
4. Every positional argument starting at this one will be a Pull Reuqest ID of
88+
a Pull Request with commit-queue set.
89+
90+
The script will iterate over the pull requests. `ncu-ci` is used to check if
91+
the last CI is still pending, and calls to the GitHub API are used to check if
92+
the PR is waiting for CI to start (`request-ci` label). The PR is skipped if CI
93+
is pending. No other CI validation is done here since `git node land` will fail
94+
if the last CI failed.
95+
96+
The script removes the `commit-queue` label. It then runs `git node land`,
97+
forwarding stdout and stderr to a file. If any errors happens,
98+
`git node land --abort` is run, and then a `commit-queue-failed` label is added
99+
to the PR, as well as a comment with the output of `git node land`.
100+
101+
If no errors happen during `git node land`, the script will use the
102+
`GITHUB_TOKEN` to push the changes to `master`, and then will leave a
103+
`Landed in ...` comment in the PR, and then will close it. Iteration continues
104+
until all PRs have done the steps above.
105+
106+
## Reverting Broken Commits
107+
108+
Reverting broken commits is done manually by Collaborators, just like when
109+
commits are landed manually via `git node land`. An easy way to revert is a
110+
good feature for the project, but is not explicitly required for the Commit
111+
Queue to work because the Action lands PRs just like collaborators do today. If
112+
once we start using the Commit Queue we notice that the number of required
113+
reverts increases drastically, we can pause the queue until a Revert Queue is
114+
implemented, but until then we can enable the Commit Queue and then work on a
115+
Revert Queue as a follow up.

‎tools/actions/commit-queue.sh

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
#!/bin/bash
2+
3+
set -xe
4+
5+
OWNER=$1
6+
REPOSITORY=$2
7+
GITHUB_TOKEN=$3
8+
shift 3
9+
10+
API_URL=https://api.github.com
11+
COMMIT_QUEUE_LABEL='commit-queue'
12+
COMMIT_QUEUE_FAILED_LABEL='commit-queue-failed'
13+
14+
function issueUrl() {
15+
echo "$API_URL/repos/${OWNER}/${REPOSITORY}/issues/${1}"
16+
}
17+
18+
function labelsUrl() {
19+
echo "$(issueUrl "${1}")/labels"
20+
}
21+
22+
function commentsUrl() {
23+
echo "$(issueUrl "${1}")/comments"
24+
}
25+
26+
function gitHubCurl() {
27+
url=$1
28+
method=$2
29+
shift 2
30+
31+
curl -fsL --request "$method" \
32+
--url "$url" \
33+
--header "authorization: Bearer ${GITHUB_TOKEN}" \
34+
--header 'content-type: application/json' "$@"
35+
}
36+
37+
38+
# TODO(mmarchini): should this be set with whoever added the label for each PR?
39+
git config --local user.email "github-bot@iojs.org"
40+
git config --local user.name "Node.js GitHub Bot"
41+
42+
for pr in "$@"; do
43+
# Skip PR if CI was requested
44+
if gitHubCurl "$(labelsUrl "$pr")" GET | jq -e 'map(.name) | index("request-ci")'; then
45+
echo "pr ${pr} skipped, waiting for CI to start"
46+
continue
47+
fi
48+
49+
# Skip PR if CI is still running
50+
if ncu-ci url "https://github.com/${OWNER}/${REPOSITORY}/pull/${pr}" 2>&1 | grep "^Result *PENDING"; then
51+
echo "pr ${pr} skipped, CI still running"
52+
continue
53+
fi
54+
55+
# Delete the commit queue label
56+
gitHubCurl "$(labelsUrl "$pr")"/"$COMMIT_QUEUE_LABEL" DELETE
57+
58+
git node land --yes "$pr" >output 2>&1 || echo "Failed to land #${pr}"
59+
# cat here otherwise we'll be supressing the output of git node land
60+
cat output
61+
62+
# TODO(mmarchini): workaround for ncu not returning the expected status code,
63+
# if the "Landed in..." message was not on the output we assume land failed
64+
if ! tail -n 10 output | grep '. Post "Landed in .*/pull/'"${pr}"; then
65+
gitHubCurl "$(labelsUrl "$pr")" POST --data '{"labels": ["'"${COMMIT_QUEUE_FAILED_LABEL}"'"]}'
66+
67+
jq -n --arg content "<details><summary>Commit Queue failed</summary><pre>$(cat output)</pre></details>" '{body: $content}' > output.json
68+
cat output.json
69+
70+
gitHubCurl "$(commentsUrl "$pr")" POST --data @output.json
71+
72+
rm output output.json
73+
# If `git node land --abort` fails, we're in unknown state. Better to stop
74+
# the script here, current PR was removed from the queue so it shouldn't
75+
# interfer again in the future
76+
git node land --abort --yes
77+
else
78+
rm output
79+
git push origin master
80+
81+
gitHubCurl "$(commentsUrl "$pr")" POST --data '{"body": "Landed in '"$(git rev-parse HEAD)"'"}'
82+
83+
gitHubCurl "$(issueUrl "$pr")" PATCH --data '{"state": "closed"}'
84+
fi
85+
done

0 commit comments

Comments
 (0)
Please sign in to comment.