Create github-actions-usage lambda

[akash1810]

The contents field of the github_workflows table contains the contents of a repository's GitHub Workflows, as a YAML string. Querying YAML in SQL isn't trivial.

What does this change?

This change adds a new table guardian_github_actions_usage, to track the Actions we're using. It has the schema:

CREATE TABLE guardian_github_actions_usage (
    evaluated_on TIMESTAMP(3) NOT NULL,
    full_name TEXT NOT NULL,
    workflow_path TEXT NOT NULL,
    workflow_uses TEXT[] NOT NULL
);

For ease, a view view_github_actions is also created, which yields results similar to¹:

full_name	workflow_path	action_name	version
guardian/service-catalogue	.github/workflows/ci.yml	actions/checkout	v4
guardian/service-catalogue	.github/workflows/ci.yml	actions/setup-node	v4
guardian/service-catalogue	.github/workflows/snyk.yml	guardian/.github/.github/workflows/sbt-node-snyk.yml	main

The guardian_github_actions_usage table is populated by a new AWS Lambda, triggered once the GitHubRepositories ECS task completes successfully.

The Lambda works as follows:

1. Get all records from the github_workflows table
2. For each record returned, parse the contents field
   1. If the field is empty, log a message
   2. Use GitHub's @actions/workflow-parser to parse the contents field²
   3. If validation fails, log a message
3. Extract the uses string from the contents
4. Save results to database

Validating the Workflow content against the schema allows us to type-cast the rows with confidence, and without any complex parsing.

The Lambda is invoked when the GitHubRepositories task successfully ends.

Note

• This change requires a database migration
• We should monitor the logs, and reduce the number of validation errors over time
• We're not filtering out archived repositories

Why?

To observe how we're using GitHub Actions across the organisation, and how it compares to the guidance provided by GitHub.

Other usage for this data includes answering:

• Which repositories are using guardian/actions-riff-raff, and which version
• Which repositories are using deprecated actions (e.g. https://github.com/guardian/actions-setup-node)
• How to setup Actions usage limits at the organisation level

How has it been verified?

• Ran locally
• Deployed to CODE, started the GitHubRepositories task, observed lambda starting once the task completes, and the new table being populated

Notes

Comparing @actions/workflow-parser to Schema Store

Previous commits in this branch used Schema Store to parse the contents field of the github_workflows table.

Schema Store is a community maintained set of YAML schemas. It's also used by JetBrains IDEs (and others, I imagine!).

@actions/workflow-parser is used within the GitHub Actions VS Code extension. In theory, @actions/workflow-parser will be kept up to date. It's unclear if this library is intended for external use, or if its an internal library. However, it's on NPM...

Its worth noting Schema Store and @actions/workflow-parser yield slightly different results. At present, the github_workflows table has 1315 rows. Schema Store can process 1305 of them, and @actions/workflow-parser can process 1301 of them.

Additionally, Schema Store and @actions/workflow-parser have differing results for some files:

• The ecs-publish.yml workflow in guardian/service-catalogue
  • Schema Store fails to parse
  • @actions/workflow-parser able to parse
• The ci.yml workflow in guardian/grid
  • Schema Store able to parse
  • @actions/workflow-parser fails to parse

I'll create an issue on the two repositories to ask for clarification here.

The default error messages provided by @actions/workflow-parser are more helpful, describing the line and column of an error:

Failed to parse workflow - path:.github/workflows/ci.yml repository:guardian/grid errors:2 [
  `.github/workflows/ci.yml (Line: 82, Col: 13): Unexpected symbol: '"'. Located at position 1 within expression: "! github.event.pull_request.head.repo.fork"`,
  `.github/workflows/ci.yml (Line: 87, Col: 13): Unexpected symbol: '"'. Located at position 1 within expression: "! github.event.pull_request.head.repo.fork"`
]

Full comparison

[!NOTE]
I've since dropped the memory size of the lambda to 512MB, as 1024MB was evidently too much. These statistics are not exact anymore, but the trend still holds.

Metric	Schema Store	@actions/workflow-parser
Duration	178020.30 ms	6453.02 ms
Billed Duration	178021 ms	6454 ms
Memory Size	1024 MB	1024 MB
Max Memory Used	256 MB	243 MB
Init Duration	369.99 ms	286.61 ms

Alternative solutions

Just use SQL

An alternative to this change would be to add a contents_as_json column to the CloudQuery table - cloudquery/cloudquery#16846.

However, the schema of a Workflow file (specifically where the uses property can appear) is tricky. It can appear within steps, or outside.

That is, the SQL to extract these values would be complicated (we'd likely continue to abstract it away into a view though).

Footnotes

1. Some columns removed, for brevity ↩

2. https://github.com/actions/languageservices/tree/main/workflow-parser is written by GitHub, and used by various GitHub authored GitHub VS Code extensions ↩

[github-actions]

Deploy build 3767 of deploy::service-catalogue to CODE

All deployment options

• Deploy build 3767 of deploy::service-catalogue to CODE
• Deploy parts of build 3767 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[adamnfish]

Looks great! Super useful feature, and I really like the code style overall. Separating side effects from logic make it easier to test the logic, perhaps we could take advantaghe of this with a couple more tests?

What's the behaviour we want when any part of this fails? Not something to worry about in this PR, but do we want best efforts for the data that it can process, or do we want it to stop completely? In either case, how would we find out that this happened, and resolve it?

I'm also a bit nervous about the schema validation, because it looks like a potential source of failures in the future. There are some comments about this below.

Great stuff though, really exciting to have this data.

[chrislomaxjones]

Awesome work!

[chrislomaxjones]

Great stuff - this is going to be really useful!

[akash1810]

Rollout steps:

• Perform database migration
• Create database user
• Manually run the lambda, to observe the lambda working
• Manually run the GitHubRepositories ECS task, to trigger the lambda, to observe the infrastructure working

[tjsilver]

This is great - thanks @akash1810 !