Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add automation for updating OpenSSL dependency #45605

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
70 changes: 70 additions & 0 deletions .github/workflows/update-openssl.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: OpenSSL update
on:
schedule:
# Run once a week at 00:05 AM UTC on Sunday.
- cron: 5 0 * * 0

workflow_dispatch:

permissions:
contents: read

jobs:
openssl-update:
if: github.repository == 'nodejs/node'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
persist-credentials: false
- name: Check if update branch already exists
run: |
BRANCH_EXISTS=$(git ls-remote --heads origin actions/tools-update-openssl)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this block the automatic backport. We won't be able to update v14 and v16 at same time considering the race condition of the branch creation.

So, I'll include the labels dont-land-on-vX and after landing you will need to manually backport and change the branch name to actions/tools-update-openssl-vX.

Does that make sense to you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same logic used for the other deps. See this line from the current dependency update action:

branch: actions/tools-update-${{ matrix.id }} # Custom branch *just* for this Action.

As far as I understand, this would be already an issue for corepack, eslint, etc right? Since the branches created for them are called tools-update-corepack and tools-update-eslint without mentioning the Node version.

Do you know how those are handled currently?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we have different versions of eslint/corepack in each release line. Could you check if the bot has ever opened PR targeting other branches instead of main?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bot will not because the automatic scheduled runs only happen on the default branch:
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule

Scheduled workflows run on the latest commit on the default or base branch.

echo "BRANCH_EXISTS=$BRANCH_EXISTS" >> $GITHUB_ENV
- name: Check and download new OpenSSL version
# Only run rest of the workflow if the update branch does not yet exist
if: ${{ env.BRANCH_EXISTS == '' }}
run: |
NEW_VERSION=$(gh api repos/quictls/openssl/releases -q '.[].tag_name|select(contains("openssl-3"))|ltrimstr("openssl-")' | head -n1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this also blocks the automatic backport to v14.x. We use the original repo on v14.x

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does v14.x have any existing action for dependency updates? I could not find any

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently there is no automation via Actions for dependency updates for any version of Node.js outside of the main branch.

NEW_VERSION_NO_RELEASE_1=$(case $NEW_VERSION in *quic1) echo ${NEW_VERSION%1};; *) echo $NEW_VERSION;; esac)
VERSION_H="./deps/openssl/config/archs/linux-x86_64/asm/include/openssl/opensslv.h"
CURRENT_VERSION=$(grep "OPENSSL_FULL_VERSION_STR" $VERSION_H | sed -n "s/^.*VERSION_STR \"\(.*\)\"/\1/p")
if [ "$NEW_VERSION_NO_RELEASE_1" != "$CURRENT_VERSION" ]; then
echo "NEW_VERSION=$NEW_VERSION" >> $GITHUB_ENV
RafaelGSS marked this conversation as resolved.
Show resolved Hide resolved
echo "HAS_UPDATE=true" >> $GITHUB_ENV
./tools/dep_updaters/update-openssl.sh download "$NEW_VERSION"
fi
env:
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }}
- name: Create PR with first commit
if: env.HAS_UPDATE
uses: gr2m/create-or-update-pull-request-action@v1
# Creates a PR with the new OpenSSL source code committed
env:
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }}
with:
author: Node.js GitHub Bot <github-bot@iojs.org>
body: This is an automated update of OpenSSL to ${{ env.NEW_VERSION }}.
branch: actions/tools-update-openssl # Custom branch *just* for this Action.
commit-message: 'deps: upgrade openssl sources to quictls/openssl-${{ env.NEW_VERSION }}'
labels: dependencies
title: 'deps: update OpenSSL to ${{ env.NEW_VERSION }}'
path: deps/openssl
- name: Regenerate platform specific files
if: env.HAS_UPDATE
run: |
sudo apt install -y nasm libtext-template-perl
./tools/dep_updaters/update-openssl.sh regenerate
env:
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }}
- name: Add second commit
# Adds a second commit to the PR with the generated platform-dependent files
if: env.HAS_UPDATE
uses: gr2m/create-or-update-pull-request-action@v1
env:
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }}
with:
author: Node.js GitHub Bot <github-bot@iojs.org>
branch: actions/tools-update-openssl # Custom branch *just* for this Action.
commit-message: 'deps: update archs files for openssl-${{ env.NEW_VERSION }}'
path: deps/openssl
41 changes: 41 additions & 0 deletions tools/dep_updaters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,45 @@ been created with the changes), do the following:
4. Create a commit for the update and in the commit message include the
important/relevant items from the changelog.

## OpenSSL

The `update-openssl.sh` script automates the steps described in
[`maintaining-openssl.md`][]. The main difference is that the script downloads
the release tarball from GitHub, instead of cloning the repo and using that as
the source code. This is useful since the release tarball does not include
development-specific files and directories (e.g the `.github` folder).

The script has to be run in two steps. The first one (using the `download`
sub-command) replaces the OpenSSL source code with the new version. The second
one (using the `regenerate` sub-command) regenerates the platform-specific
files. This makes it easier to create two separate git commits, making the git
history more descriptive.

For example, in order to update to version `3.0.7+quic1`, the following commands
should be run:

```bash
./tools/dep_updaters/update-openssl.sh download 3.0.7+quic1
git add -A deps/openssl/openssl
git commit -m "deps: upgrade openssl sources to quictls/openssl-3.0.7+quic1"

./tools/dep_updaters/update-openssl.sh regenerate 3.0.7+quic1
git add -A deps/openssl/config/archs deps/openssl/openssl
git commit -m "deps: update archs files for openssl"
```

Once the script has run (either manually, or by CI in which case a PR will have
been created with the changes), do the following:

1. Check the `CHANGES.md` file in the [repo](https://github.com/quictls/openssl)
for things that might require changes in Node.js.
2. Check the diffs to ensure the changes are right. Even if there are no changes
in the source, `buildinf.h` files will be updated because they have timestamp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to this PR, but we should add a task somwhere to see if we can address this. Having it be more repeatable would be good.

data in them.
3. Check that Node.js compiles without errors and the tests pass.
4. Create a commit for the update and in the commit message include the
important/relevant items from the changelog.

## postject

The `update-postject.sh` script downloads postject from the [npm package](http://npmjs.com/package/postject)
Expand All @@ -80,3 +119,5 @@ been created with the changes), do the following:
2. Check that Node.js compiles without errors and the tests pass.
3. Create a commit for the update and in the commit message include the
important/relevant items from the changelog.

[`maintaining-openssl.md`]: https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-openssl.md
99 changes: 99 additions & 0 deletions tools/dep_updaters/update-openssl.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
#!/bin/sh
set -e
# Shell script to update OpenSSL in the source tree to a specific version
# Based on https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-openssl.md

cleanup() {
EXIT_CODE=$?
[ -d "$WORKSPACE" ] && rm -rf "$WORKSPACE"
exit $EXIT_CODE
}

download() {
if [ -z "$1" ]; then
echo "Error: please provide an OpenSSL version to update to"
echo " e.g. ./$0 download 3.0.7+quic1"
exit 1
fi

OPENSSL_VERSION=$1
echo "Making temporary workspace..."
WORKSPACE=$(mktemp -d 2> /dev/null || mktemp -d -t 'tmp')


cd "$WORKSPACE"

echo "Fetching OpenSSL source archive..."
curl -sL "https://api.github.com/repos/quictls/openssl/tarball/openssl-$OPENSSL_VERSION" | tar xzf -
mv quictls-openssl-* openssl

echo "Replacing existing OpenSSL..."
rm -rf "$DEPS_DIR/openssl/openssl"
mv "$WORKSPACE/openssl" "$DEPS_DIR/openssl/"

echo "All done!"
echo ""
echo "Please git add openssl, and commit the new version:"
echo ""
echo "$ git add -A deps/openssl/openssl"
echo "$ git commit -m \"deps: upgrade openssl sources to quictls/openssl-$OPENSSL_VERSION\""
echo ""
}

regenerate() {
command -v perl >/dev/null 2>&1 || { echo >&2 "Error: 'Perl' required but not installed."; exit 1; }
command -v nasm >/dev/null 2>&1 || { echo >&2 "Error: 'nasm' required but not installed."; exit 1; }
command -v as >/dev/null 2>&1 || { echo >&2 "Error: 'GNU as' required but not installed."; exit 1; }
perl -e "use Text::Template">/dev/null 2>&1 || { echo >&2 "Error: 'Text::Template' Perl module required but not installed."; exit 1; }

echo "Regenerating platform-dependent files..."

make -C "$DEPS_DIR/openssl/config" clean
# Needed for compatibility with nasm on 32-bit Windows
# See https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-openssl.md#2-execute-make-in-depsopensslconfig-directory
sed -i 's/#ifdef/%ifdef/g' "$DEPS_DIR/openssl/openssl/crypto/perlasm/x86asm.pl"
sed -i 's/#endif/%endif/g' "$DEPS_DIR/openssl/openssl/crypto/perlasm/x86asm.pl"
make -C "$DEPS_DIR/openssl/config"

echo "All done!"
echo ""
echo "Please commit the regenerated files:"
echo ""
echo "$ git add -A deps/openssl/config/archs deps/openssl/openssl"
echo "$ git commit -m \"deps: update archs files for openssl\""
echo ""
}

help() {
echo "Shell script to update OpenSSL in the source tree to a specific version"
echo "Sub-commands:"
printf "%-23s %s\n" "help" "show help menu and commands"
printf "%-23s %s\n" "download" "download and replace OpenSSL source code with new version"
printf "%-23s %s\n" "regenerate" "regenerate platform-specific files"
echo ""
exit "${1:-0}"
}

main() {
if [ ${#} -eq 0 ]; then
help 0
fi

trap cleanup INT TERM EXIT

BASE_DIR=$(cd "$(dirname "$0")/../.." && pwd)
DEPS_DIR="$BASE_DIR/deps"

case ${1} in
help | download | regenerate )
$1 "${2}"
;;
* )
echo "unknown command: $1"
help 1
exit 1
;;
esac
}

main "$@"