Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create github-actions-usage lambda #810

Merged
merged 25 commits into from
Feb 27, 2024
Merged

Create github-actions-usage lambda #810

merged 25 commits into from
Feb 27, 2024

Commits on Feb 27, 2024

  1. feat(github-actions-usage): Initial structure

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    5a87904 View commit details
    Browse the repository at this point in the history

  2. feat(github-actions-usage): Add Prisma querying

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    147a35f View commit details
    Browse the repository at this point in the history

  3. feat(github-actions-usage): Validate GitHub Workflow content against … 

    …the schema

Validating the Workflow content against the schema allows us to type-cast the objects with confidence.

Using a schema from https://www.schemastore.org.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    46c355e View commit details
    Browse the repository at this point in the history

  4. feat(github-actions-usage): Define types

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    8998f2f View commit details
    Browse the repository at this point in the history

  5. refactor(github-actions-usage): Move queries to own file

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    9be099c View commit details
    Browse the repository at this point in the history

  6. feat(github-actions-usage): Get repository full name

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    937e54a View commit details
    Browse the repository at this point in the history

  7. feat(github-actions-usage): Save results to database table `guardian_… 

    …github_actions_usage`
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    9b8ab53 View commit details
    Browse the repository at this point in the history

  8. refactor(github-actions-usage): Join tables to reduce checks 

    By joining `github_workflows` with `github_repositories` at query time,
the volume of code reduces, due to less validation being necessary.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    90a517d View commit details
    Browse the repository at this point in the history

  9. fix(github-actions-usage): Handle YAML parse errors

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    9b6d7a9 View commit details
    Browse the repository at this point in the history

  10. refactor(github-actions-usage): Split data logic across files 

    Organise the logic into three distinct steps:
1. Read data
2. Transform data
3. Write data

This should improve readability.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    a678f49 View commit details
    Browse the repository at this point in the history

  11. refactor(github-actions-usage): Interface renames for clarity 

    Also add comments.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    8c463bd View commit details
    Browse the repository at this point in the history

  12. chore(github-actions-usage): Log validation error details

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    3bec6e7 View commit details
    Browse the repository at this point in the history

  13. test(github-actions-usage): Add tests for validateRawWorkflow

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    9e1e3b5 View commit details
    Browse the repository at this point in the history

  14. refactor(github-actions-usage): Clear github_actions_usage table mo… 

    …re explicitly
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    5933ed2 View commit details
    Browse the repository at this point in the history

  15. refactor(github-actions-usage): Remove side-effect

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    b598cb3 View commit details
    Browse the repository at this point in the history

  16. fix(github-actions-usage): Delete and save results in a single transa… 

    …ction
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    154cb19 View commit details
    Browse the repository at this point in the history

  17. feat(github-actions-usage): Create database user github_actions_usage

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    d3d2459 View commit details
    Browse the repository at this point in the history

  18. feat(github-actions-usage): Add view_github_actions SQL view 

    This view shows the archived status of a repository, the name of the Action being used, and the version.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    3c20a37 View commit details
    Browse the repository at this point in the history

  19. ci: Compare GitHub Workflow schema with upstream

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    061ab4b View commit details
    Browse the repository at this point in the history

  20. feat(github-actions-usage): Add AWS Lambda 

    Triggered when the `GitHubRepositories` task stops successfully.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e02d05d View commit details
    Browse the repository at this point in the history

  21. docs(github-actions-usage): Add README

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    c7d4fa2 View commit details
    Browse the repository at this point in the history

  22. refactor(github-actions-usage): Parse workflows using `@actions/workf… 

    …low-parser`

Swap the community maintained Schema Store parsing of GitHub Workflow files
for the GitHub authored `@actions/workflow-parser` NPM module.

Interestingly, the two approaches yield slightly different results:
- https://github.com/guardian/service-catalogue/blob/3f25b6c553e1ed2b192bce11cc15e6f25afa7239/.github/workflows/ecs-publish.yml
  - Schema Store fails to parse
  - `@actions/workflow-parser` able to parse
- https://github.com/guardian/grid/blob/e37e3acaeea198beb82896a143cc9c66d200aadc/.github/workflows/ci.yml
  - Schema Store able to parse
  - `@actions/workflow-parser` fails to parse

Anecdotally, this version also appears to be more performant.

`@actions/workflow-parser` is an ESM only module, and it was proving tricky to get Jest to work with it.
For this reason, the tests use Node's native test runner.
    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    c1c910b View commit details
    Browse the repository at this point in the history

  23. refactor(github-actions-usage): Rename for clarity

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    18e7c02 View commit details
    Browse the repository at this point in the history

  24. test: Add tests for removeUndefined function

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    fdc0c63 View commit details
    Browse the repository at this point in the history

  25. refactor(github-actions-usage): Simplify boolean logic

    @akash1810
    akash1810 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    4a77c28 View commit details
    Browse the repository at this point in the history