Skip to content

Commit

Permalink
refactor(github-actions-usage): Parse workflows using `@actions/workf…
Browse files Browse the repository at this point in the history
…low-parser`

Swap the community maintained Schema Store parsing of GitHub Workflow files
for the GitHub authored `@actions/workflow-parser` NPM module.

Interestingly, the two approaches yield slightly different results:
- https://github.com/guardian/service-catalogue/blob/3f25b6c553e1ed2b192bce11cc15e6f25afa7239/.github/workflows/ecs-publish.yml
  - Schema Store fails to parse
  - `@actions/workflow-parser` able to parse
- https://github.com/guardian/grid/blob/e37e3acaeea198beb82896a143cc9c66d200aadc/.github/workflows/ci.yml
  - Schema Store able to parse
  - `@actions/workflow-parser` fails to parse

Anecdotally, this version also appears to be more performant.

`@actions/workflow-parser` is an ESM only module, and it was proving tricky to get Jest to work with it.
For this reason, the tests use Node's native test runner.
  • Loading branch information
akash1810 committed Feb 26, 2024
1 parent 302cb67 commit a176033
Show file tree
Hide file tree
Showing 13 changed files with 289 additions and 286 deletions.
5 changes: 0 additions & 5 deletions jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,5 @@ module.exports = {
transform,
testMatch: ['<rootDir>/packages/snyk-integrator/**/*.test.ts'],
},
{
displayName: 'github-actions-usage',
transform,
testMatch: ['<rootDir>/packages/github-actions-usage/**/*.test.ts'],
},
],
};
63 changes: 33 additions & 30 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 6 additions & 10 deletions packages/github-actions-usage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This app, deployed as an AWS Lambda, tracks which GitHub Actions are in use acro
1. Get all records from the `github_workflows` table
2. For each record returned, parse the `contents` field
1. If the field is empty, log a message
2. Use [Ajv](https://ajv.js.org/) to validate the `contents` field against a [schema](https://json.schemastore.org/github-workflow.json) published by https://www.schemastore.org/json/
2. Use GitHub's [`@actions/workflow-parser`](https://github.com/actions/languageservices/tree/main/workflow-parser) to parse the `contents` field
3. If validation fails, log a message
3. Extract the `uses` string from the `contents`
4. Save results to database
Expand All @@ -18,12 +18,8 @@ The [`view_github_actions` SQL view](../common/prisma/migrations/20240222201635_
to make it easier to query the data.
This view shows the archived status of a repository, the name of the Action being used, and the version.

## Updating the schema
The [schema](./src/schema/github-workflow.json) might need updating from time to time.

To do this, run the following, and commit the resulting file:

```bash
cd /path/to/github-actions-usage
wget --output-document src/schema/github-workflow.json https://json.schemastore.org/github-workflow.json
```
## Testing
The tests are written with [Node's test runner](https://nodejs.org/api/test.html).
This is in contrast to other packages in this repository which use Jest.
This is because Jest had issues with the `@actions/workflow-parser` package;
Jest was failing with "cannot find module" errors.
6 changes: 3 additions & 3 deletions packages/github-actions-usage/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
"name": "github-actions-usage",
"version": "0.0.0",
"scripts": {
"test": "jest --detectOpenHandles --config ../../jest.config.js --selectProjects github-actions-usage",
"test": "node --import tsx --test **/*.test.ts",
"start": "APP=github-actions-usage tsx src/run-locally.ts",
"prebuild": "rm -rf dist",
"build": "esbuild src/index.ts --bundle --platform=node --target=node20 --outdir=dist --external:@prisma/client --external:prisma"
},
"type": "module",
"dependencies": {
"ajv": "8.12.0",
"yaml": "2.3.4"
"@actions/workflow-parser": "0.3.8"
}
}
2 changes: 1 addition & 1 deletion packages/github-actions-usage/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export async function main(...args: unknown[]) {

const now = new Date();
const data = await getWorkflows(prismaClient);
const recordsToSave = transform(data);
const recordsToSave = await transform(data);

/*
Prisma performs `deleteMany` and `createMany` in separate transactions by default.
Expand Down
4 changes: 1 addition & 3 deletions packages/github-actions-usage/src/run-locally.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,4 @@ import { main } from './index';
// Read the .env file from the repository root
config({ path: `../../.env` });

if (require.main === module) {
void main();
}
void main();

0 comments on commit a176033

Please sign in to comment.