Cloud Runner v0 - Reliable and trimmed down cloud runner

[frostebite]

This pull request implements the minimum features to implement the promise of cloud runner fully either via the default parameters or via the extensible override parameters.

Release Status Changes

K8s support

unstable/development->experimental
You can in theory easily accomplish distributed caching via pull cache override for k8s in this release to make k8s a fully viable workflow for massive projects. Will be experimenting with some multi-shared read/write PVs from Rook in the next PR.

Cloud Runner General Support

experimental->preview
AWS implementation out of the box can handle massive projects with sensible parameters.

Preview phase of cloud runner

If accepted I will begin using tag releases to communicate stable releases for cloud runner. cloud-runner-develop will continue as the primary integration branch.

The preview phase of the cloud runner release will represent:

• further feedback gathering
• alignment with any needed refinement for the wider game ci project.
• any needed steps to simplify and align with beginner-friendly promise of game-ci, including but not limited to documentation improvements.
• garbage collection improvements (guarantees, check active resources)
• Moving away from using the branch as reference for cloud runner. Instead, we will use SHA. Currently, we jump to the latest commit on a given branch.
• Integrated ephemeral runner support to cut-down impact of long run times with cloud runner.

Post preview I will explore warm-starts, diff-based runs, generic dependency startup/cleanup hooks, editor integration, extensible logging, gcp run, testing improvements (including tests for caching hit/miss specific performance tests) extensible logging, benchmarking (large reference project), better artifact handling, more explicit checking of build success to make it convenient to expose results into output and probably more...

When satisfied with the above cloud runner will progress to released/ga.

Other improvements

There is also a general refactoring to make cloud runner polished and shiny. For example:

• Many files and functions were removed (such as cloud runner state, which was simplified into a smaller implementation in cloud runner's main file) as part of a general simplification and cutting down on structures that didn't end up having solid responsibilities.
• All async input has been removed and generally cleaned up to better align with the build parameters pattern.
• Fully realized CLI support
• Override input source - use terminal commands to read any input source to make it easy to specify large values/secrets such as a license or steps YAML
• Hooks - (pre/post) (setup/build) container steps and (pre/post/all) (setup/build) terminal command hooks.
• Combined setup/run cloud runner container to minimize startup time.
• Some important increases in test coverage (quick verification of caching and deep verification of caching)

[github-actions]

[codecov-commenter]

Codecov Report

Merging #353 (9f42bc9) into main (2b399b2) will increase coverage by 9.76%.
The diff coverage is 59.33%.

@@            Coverage Diff             @@
##             main     #353      +/-   ##
==========================================
+ Coverage   44.92%   54.69%   +9.76%     
==========================================
  Files          51       62      +11     
  Lines        1487     1918     +431     
  Branches      259      316      +57     
==========================================
+ Hits          668     1049     +381     
- Misses        779      865      +86     
+ Partials       40        4      -36     

Impacted Files	Coverage Δ
src/model/cloud-runner/cloud-runner-step-state.ts	100.00% <ø> (ø)
...el/cloud-runner/providers/k8s/kubernetes-secret.ts	23.07% <ø> (ø)
...runner/providers/k8s/kubernetes-service-account.ts	22.22% <ø> (ø)
...odel/cloud-runner/providers/aws/aws-task-runner.ts	8.03% <14.70%> (ø)
...l/cloud-runner/providers/k8s/kubernetes-storage.ts	10.52% <20.00%> (ø)
src/model/cloud-runner/remote-client/index.ts	20.63% <20.63%> (ø)
...cloud-runner/services/depdency-override-service.ts	25.00% <25.00%> (ø)
...oud-runner/services/cloud-runner-query-override.ts	26.47% <26.47%> (ø)
...-runner/providers/aws/commands/aws-cli-commands.ts	30.00% <30.00%> (ø)
src/model/cli/cli.ts	31.34% <30.61%> (ø)
... and 57 more

[webbertakken]

Amazing work once again. A huge step forward!