ci: GitHub Actions to validate a Maven BOM (#5928)

Fixes #5922 It worked in https://togithub.com/googleapis/gapic-generator-java/pull/1629 Todo: - Merge this pull request 5928. - Update the branch name in https://togithub.com/googleapis/gapic-generator-java/pull/1629 to "main" - Add another check in https://togithub.com/googleapis/google-cloud-java with "main"
googleapis · Apr 17, 2023 · 04160b1 · 04160b1
1 parent 28f172d
commit 04160b1
Show file tree

Hide file tree

Showing 9 changed files with 342 additions and 10 deletions.
diff --git a/.github/workflows/ci-release-note-generation.yaml b/.github/workflows/ci-release-note-generation.yaml
@@ -10,8 +10,9 @@ jobs:
     - uses: actions/checkout@v3
     - uses: actions/setup-java@v3
       with:
-        distribution: zulu
+        distribution: temurin
         java-version: 11
+        cache: maven
     - run: java -version
     - name: Run test in release-note-generation
       shell: bash
@@ -27,8 +28,9 @@ jobs:
     - uses: actions/checkout@v3
     - uses: actions/setup-java@v3
       with:
-        distribution: zulu
+        distribution: temurin
         java-version: 11
+        cache: maven
     - run: java -version
     - name: Dry-run release-note-generation
       shell: bash

diff --git a/.github/workflows/dashboard.yaml b/.github/workflows/dashboard.yaml
@@ -11,8 +11,9 @@ jobs:
     - uses: actions/checkout@v3
     - uses: actions/setup-java@v3
       with:
-        distribution: zulu
+        distribution: temurin
         java-version: 8
+        cache: maven
     - run: java -version
     - run: .kokoro/dashboard.sh
       env:
@@ -24,8 +25,9 @@ jobs:
     - uses: actions/checkout@v3
     - uses: actions/setup-java@v3
       with:
-        distribution: zulu
+        distribution: temurin
         java-version: 8
+        cache: maven
     - run: java -version
     - run: .kokoro/dashboard.sh
       env:

diff --git a/.github/workflows/full-convergence-check.yaml b/.github/workflows/full-convergence-check.yaml
@@ -12,16 +12,17 @@ jobs:
     - uses: actions/checkout@v3
     - uses: actions/setup-java@v3
       with:
-        distribution: zulu
-        java-version: 8
+        distribution: temurin
+        java-version: 11
+        cache: maven
     - run: java -version
     - name: Install BOMs
       run: |
         mvn -B -V -ntp install
     - name: Ensure the members of the Libraries BOM exist in Maven Central
-      run: |
-        mvn -B -V -ntp verify -Dtest="BomContentTest#testLibrariesBomReachable"
-      working-directory: tests
+      uses: ./tests/validate-bom
+      with:
+        bom-path: libraries-bom/pom.xml
     - name: Ensure the BOM has valid content (at releases)
       if: github.head_ref == 'release-please--branches--main'
       run: |

diff --git a/.github/workflows/release-note-generation.yaml b/.github/workflows/release-note-generation.yaml
@@ -17,8 +17,9 @@ jobs:
     - uses: actions/checkout@v3
     - uses: actions/setup-java@v3
       with:
-        distribution: zulu
+        distribution: temurin
         java-version: 11
+        cache: maven
     - run: java -version
     - name: Pick Libraries BOM version
       id: pick-version

diff --git a/tests/validate-bom/README.md b/tests/validate-bom/README.md
@@ -0,0 +1,80 @@
+# Validate Maven BOM GitHub Action
+
+This action validates a [Maven BOM](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms
+) specified as argument.
+
+This action performs the following steps:
+
+- It reads the BOM and gets all artifacts.
+  - It may filter out "testlib" artifacts if they cause problems in subsequent steps
+- It creates a canary Maven project (a directory with a pom.xml file) with the artifacts as the dependencies.
+  The canary project uses the BOM and declares the artifacts in the BOM as dependencies.
+- It runs `mvn install` in the canary project.
+  If the BOM is valid, it should fetch dependencies (the artifacts in the BOM) without an error.
+
+## Usage
+
+You can use this action via `uses: googleapis/java-cloud-bom/tests/validate-bom@main`
+in one of the steps in a job in your GitHub repository.
+
+Note that before running this action the caller needs to make the BOM and its
+listing artifacts available in Maven Central or local Maven repository.
+
+Here is a concrete example to define a job to use this "validate-bom" action in
+a GitHub Actions workflow file:
+
+```
+  validate-bom:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - uses: actions/setup-java@v3
+      with:
+        java-version: 11
+        distribution: temurin
+        cache: maven
+    - name: Install Maven artifacts locally
+      run: |
+        mvn install -B -ntp -DskipTests
+    - uses: googleapis/java-cloud-bom/tests/validate-bom@main
+      with:
+        path: <path_to_bom_pom.xml>
+```
+
+### Results
+
+If there's an error in building the canary project, the check fails.
+You see errors in the log:
+
+```
+[INFO] ------------------------------------------------------------------------
+[INFO] BUILD FAILURE
+[INFO] ------------------------------------------------------------------------
+[INFO] Total time:  14.253 s
+[INFO] Finished at: 2023-04-14T20:41:59Z
+[INFO] ------------------------------------------------------------------------
+Error:  Failed to execute goal on project bom-validation-canary-project: Could n
+ot resolve dependencies for project com.google.cloud:bom-validation-canary-proje
+ct:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: com.google
+.analytics.api.grpc:grpc-google-analytics-admin-v1alpha:jar:0.24.0 ...
+```
+
+In this error message, there were invalid artifacts defined in the BOM
+(wrong group IDs).
+
+If there's no error, the check passes with a successful message:
+
+```
+[INFO] Installing /tmp/bom-validation/pom.xml to /home/runner/.m2/repository/...
+[INFO] ------------------------------------------------------------------------
+[INFO] BUILD SUCCESS
+[INFO] ------------------------------------------------------------------------
+[INFO] Total time:  5.147 s
+[INFO] Finished at: 2023-04-14T20:35:58Z
+[INFO] ------------------------------------------------------------------------
+```
+
+# Disclaimer
+
+This is not an official Google product.
+This is intended for Google-internal usages only.
diff --git a/tests/validate-bom/action.yml b/tests/validate-bom/action.yml
@@ -0,0 +1,47 @@
+name: 'Maven BOM Validation'
+description: 'Validation for the content of a Maven BOM'
+inputs:
+  bom-path:
+    description: "The relative path from the repository root to the pom.xml file"
+    required: true
+runs:
+  using: "composite"
+  steps:
+  - uses: actions/setup-java@v3
+    with:
+      distribution: temurin
+      java-version: 11
+      cache: maven
+  - name: Set up Maven
+    uses: stCarolas/setup-maven@v4.5
+    with:
+      maven-version: 3.8.4
+  - name: Create temporary directory /tmp/bom-validation
+    shell: bash
+    run: mkdir -p /tmp/bom-validation
+  - name: Create a canary project that uses the BOM
+    shell: bash
+    run: |
+      if [ ! -r "${{ inputs.bom-path }}" ]; then
+        echo "The input bom-path ${{ inputs.bom-path }} is not readable"
+        exit 1
+      fi
+      
+      bom_absolute_path=$(realpath "${{ inputs.bom-path }}")
+
+      # Before this "cd", the working directory is the repository that calls
+      # this action. To use validate-bom classes, it needs to change directory
+      # to the directory that defines this action.
+      cd ${{ github.action_path }}
+
+      echo "Compiling CreateBomCanaryProject.java in $(pwd)"
+      mvn -V -ntp compile
+      echo "Running CreateBomCanaryProject with ${bom_absolute_path}"
+      mvn -V -ntp -B exec:java -DoutputPath=/tmp/bom-validation -DbomPath="${bom_absolute_path}"
+  - name: Build the canary project that uses the BOM
+    shell: bash
+    working-directory: /tmp/bom-validation
+    run: |
+      echo "working directory: $(pwd)"
+      mvn -ntp -B install
+
diff --git a/tests/validate-bom/pom.xml b/tests/validate-bom/pom.xml
@@ -0,0 +1,56 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <parent>
+    <artifactId>java-cloud-bom-root</artifactId>
+    <groupId>com.google.cloud</groupId>
+    <version>0.1.0</version>
+    <relativePath>../../</relativePath>
+  </parent>
+  <modelVersion>4.0.0</modelVersion>
+
+  <artifactId>bom-canary-project-creation</artifactId>
+
+  <name>BOM Canary Project Creation</name>
+
+  <properties>
+    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+    <maven.compiler.source>11</maven.compiler.source>
+    <maven.compiler.target>11</maven.compiler.target>
+  </properties>
+
+  <dependencies>
+    <dependency>
+      <groupId>com.google.cloud.tools</groupId>
+      <artifactId>dependencies</artifactId>
+      <version>1.5.13</version>
+    </dependency>
+    <dependency>
+      <groupId>junit</groupId>
+      <artifactId>junit</artifactId>
+      <version>4.13.1</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>com.google.truth</groupId>
+      <artifactId>truth</artifactId>
+      <version>1.1.3</version>
+      <scope>test</scope>
+    </dependency>
+  </dependencies>
+
+  <build>
+    <plugins>
+      <plugin>
+        <groupId>org.codehaus.mojo</groupId>
+        <artifactId>exec-maven-plugin</artifactId>
+        <configuration>
+          <skip>false</skip>
+          <mainClass>com.google.cloud.CreateBomCanaryProject</mainClass>
+        </configuration>
+      </plugin>
+    </plugins>
+  </build>
+</project>
diff --git a/tests/validate-bom/src/main/java/com/google/cloud/CreateBomCanaryProject.java b/tests/validate-bom/src/main/java/com/google/cloud/CreateBomCanaryProject.java
@@ -0,0 +1,129 @@
+package com.google.cloud;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import com.google.cloud.tools.opensource.dependencies.Bom;
+import com.google.cloud.tools.opensource.dependencies.MavenRepositoryException;
+import com.google.common.base.Verify;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.eclipse.aether.artifact.Artifact;
+
+/**
+ * Creates a Maven project that uses the specified BOM at the specified directory. This class reads
+ * the following system properties:
+ *
+ * <ul>
+ *   <li>outputPath: the path to the directory to create the Maven project
+ *   <li>bomPath: the path to the BOM
+ * </ul>
+ */
+public class CreateBomCanaryProject {
+
+  public static void main(String[] arguments) throws Exception {
+    String outputPathProperty = System.getProperty("outputPath");
+    checkNotNull(outputPathProperty, "System property outputPath should not be null");
+    Path outputProjectDirectory = Paths.get(outputPathProperty);
+    String bomPathProperty = System.getProperty("bomPath");
+    checkNotNull(bomPathProperty, "System property bomPath should not be null");
+    Path bomPath = Paths.get(bomPathProperty);
+
+    Bom bom;
+    try {
+      bom = Bom.readBom(bomPath);
+    } catch (MavenRepositoryException exception) {
+      throw new IOException(
+          "Could not read the BOM: "
+              + bomPath
+              + ". Please ensure all artifacts in the BOM are available in Maven Central or local"
+              + " Maven repository.",
+          exception);
+    }
+
+    String pomTemplate = readPomTemplate();
+
+    String dependencyManagementSection = calculateDependencyManagementSection(bom);
+    String dependenciesSection = calculateDependenciesSection(bom);
+
+    String replacedContent =
+        pomTemplate
+            .replace("DEPENDENCY_MANAGEMENT", dependencyManagementSection)
+            .replace("DEPENDENCIES", dependenciesSection);
+
+    Path pomToWrite = outputProjectDirectory.resolve("pom.xml");
+    Files.write(pomToWrite, replacedContent.getBytes());
+    System.out.println("Wrote " + pomToWrite);
+  }
+
+  /** Returns the pom.xml template content. */
+  private static String readPomTemplate() throws IOException {
+    try (InputStream inputStream =
+        CreateBomCanaryProject.class.getClassLoader().getResourceAsStream("template.pom.xml")) {
+      Verify.verifyNotNull(inputStream);
+      return new String(inputStream.readAllBytes());
+    }
+  }
+
+  /** Returns the dependencyManagement section to import {@code bom}. */
+  private static String calculateDependencyManagementSection(Bom bom) {
+    String[] coordinatesElements = bom.getCoordinates().split(":");
+    Verify.verify(coordinatesElements.length == 3);
+    String groupId = coordinatesElements[0];
+    String artifactId = coordinatesElements[1];
+    String version = coordinatesElements[2];
+
+    StringBuilder builder = new StringBuilder();
+    builder.append("  <dependencyManagement>\n");
+    builder.append("    <dependencies>\n");
+    builder.append("      <dependency>\n");
+    builder.append("        <groupId>").append(groupId).append("</groupId>\n");
+    builder.append("        <artifactId>").append(artifactId).append("</artifactId>\n");
+    builder.append("        <version>").append(version).append("</version>\n");
+    builder.append("        <type>pom</type>\n");
+    builder.append("        <scope>import</scope>\n");
+    builder.append("      </dependency>\n");
+    builder.append("    </dependencies>\n");
+    builder.append("  </dependencyManagement>\n");
+    return builder.toString();
+  }
+
+  /** Returns the "dependencies" section that would declare all artifacts appear in {@code bom}. */
+  private static String calculateDependenciesSection(Bom bom) {
+    StringBuilder builder = new StringBuilder();
+    builder.append("  <dependencies>\n");
+
+    for (Artifact managedDependency : bom.getManagedDependencies()) {
+      Map<String, String> properties = managedDependency.getProperties();
+      String classifier = managedDependency.getClassifier();
+      if ("tests".equals(classifier)) {
+        // Tests classifier artifacts are not for customers
+        continue;
+      }
+      String type = properties.get("type");
+      if ("pom".equals(type)) {
+        // Some artifacts have :pom" type, such as io.grpc:protoc-gen-grpc-java
+        // and com.google.api-client:google-api-client-assembly. We are only interested
+        // in "jar" artifacts.
+        continue;
+      }
+
+      builder.append("    <dependency>\n");
+      builder
+          .append("      <groupId>")
+          .append(managedDependency.getGroupId())
+          .append("</groupId>\n");
+      builder
+          .append("      <artifactId>")
+          .append(managedDependency.getArtifactId())
+          .append("</artifactId>\n");
+      builder.append("    </dependency>\n");
+    }
+    builder.append("  </dependencies>\n");
+
+    return builder.toString();
+  }
+}