Process `snapshot` tokens #3135

davidomid · 2024-02-07T17:56:26Z

Closes: https://github.com/github/hosted-runners/issues/526

Summary of changes:

Added deserialisation of a new TemplateToken called snapshot to AgentJobRequestMessage.
Added evaluation of snapshot from TemplateToken to a new Snapshot type.
Added a new SnapshotOperationProvider service, responsible for processing Snapshot objects.
Updated JobExtension to evaluate a Snapshot if one is present in the job template, then register a new post-job step for invoking SnapshotOperationProvider with it.
Added new L0 tests for JobExtension and SnapshotOperationProvider.
Updated existing tests for regression.

…ub.com/davidomid/runner into users/davidomid/process-snapshot-token

…tionProviderL0 tests

src/Runner.Worker/JobExtension.cs

ericsciple

Thanks! This makes sense to me. Although I would like to get a second review from someone who works in the runner more recently.

src/Sdk/DTPipelines/Pipelines/ObjectTemplating/PipelineTemplateConverter.cs

src/Sdk/DTPipelines/workflow-v1.0.json

src/Runner.Worker/JobExtension.cs

src/Runner.Worker/SnapshotOperationProvider.cs

ericsciple

Thanks! This makes sense to me. Will defer to Ting whether he has any remaining concerns.

…314.0 ) (#4273) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/onedr0p/actions-runner](https://ghcr.io/onedr0p/actions-runner) ([source](https://github.com/actions/runner)) | minor | `2.313.0` -> `2.314.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>actions/runner (ghcr.io/onedr0p/actions-runner)</summary> ### [`v2.314.0`](https://github.com/actions/runner/releases/tag/v2.314.0) [Compare Source](https://github.com/actions/runner/compare/v2.313.0...v2.314.0) ##### What's Changed - Prepare v2.313.0 Release by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3137 - Pass RunnerOS during job acquire. by [@TingluoHuang](https://github.com/TingluoHuang) in [actions/runner#3140 - Process `snapshot` tokens by [@davidomid](https://github.com/davidomid) in [actions/runner#3135 - Update dotnet sdk to latest version [@6](https://github.com/6).0.419 by [@github-actions](https://github.com/github-actions) in [actions/runner#3158 - handle broker run service exception handling by [@yaananth](https://github.com/yaananth) in [actions/runner#3163 - Add a retry logic to docker login operation by [@enescakir](https://github.com/enescakir) in [actions/runner#3089 - Broker fixes for token refreshes and AccessDeniedException by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3161 - Remove USE_BROKER_FLOW by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3162 - Refresh Token for BrokerServer by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3167 - Better step timeout message. by [@TingluoHuang](https://github.com/TingluoHuang) in [actions/runner#3166 ##### New Contributors - [@davidomid](https://github.com/davidomid) made their first contribution in [actions/runner#3135 - [@enescakir](https://github.com/enescakir) made their first contribution in [actions/runner#3089 **Full Changelog**: actions/runner@v2.313.0...v2.314.0 *Note: Actions Runner follows a progressive release policy, so the latest release might not be available to your enterprise, organization, or repository yet. To confirm which version of the Actions Runner you should expect, please view the download instructions for your enterprise, organization, or repository. See https://docs.github.com/en/enterprise-cloud@latest/actions/hosting-your-own-runners/adding-self-hosted-runners* ##### Windows x64 We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows. The following snipped needs to be run on `powershell`: ```powershell ##### Create a folder under the drive root mkdir \actions-runner ; cd \actions-runner ##### Download the latest runner package Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-win-x64-2.314.0.zip -OutFile actions-runner-win-x64-2.314.0.zip ##### Extract the installer Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-x64-2.314.0.zip", "$PWD") ``` ##### \[Pre-release] Windows arm64 **Warning:** Windows arm64 runners are currently in preview status and use [unofficial versions of nodejs](https://unofficial-builds.nodejs.org/). They are not intended for production workflows. We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows. The following snipped needs to be run on `powershell`: ```powershell ##### Create a folder under the drive root mkdir \actions-runner ; cd \actions-runner ##### Download the latest runner package Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-win-arm64-2.314.0.zip -OutFile actions-runner-win-arm64-2.314.0.zip ##### Extract the installer Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-arm64-2.314.0.zip", "$PWD") ``` ##### OSX x64 ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-osx-x64-2.314.0.tar.gz ##### Extract the installer tar xzf ./actions-runner-osx-x64-2.314.0.tar.gz ``` ##### OSX arm64 (Apple silicon) ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-osx-arm64-2.314.0.tar.gz ##### Extract the installer tar xzf ./actions-runner-osx-arm64-2.314.0.tar.gz ``` ##### Linux x64 ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-linux-x64-2.314.0.tar.gz ##### Extract the installer tar xzf ./actions-runner-linux-x64-2.314.0.tar.gz ``` ##### Linux arm64 ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-linux-arm64-2.314.0.tar.gz ##### Extract the installer tar xzf ./actions-runner-linux-arm64-2.314.0.tar.gz ``` ##### Linux arm ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.0/actions-runner-linux-arm-2.314.0.tar.gz ##### Extract the installer tar xzf ./actions-runner-linux-arm-2.314.0.tar.gz ``` ##### Using your self hosted runner For additional details about configuring, running, or shutting down the runner please check out our [product docs.](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/adding-self-hosted-runners) ##### SHA-256 Checksums The SHA-256 checksums for the packages included in this build are shown below: - actions-runner-win-x64-2.314.0.zip 70737a3df30ff9bc0b395a0331309aba59da760eb50da4ea2ff5cb768fa7f6e4 - actions-runner-win-arm64-2.314.0.zip e4677bbd8fa0a4fcab0491e6865cbd5cb0597eba0690ab8f5b097a15ef8d4554 - actions-runner-osx-x64-2.314.0.tar.gz 5d928a482572e84f6c823a5f340e7fddec612cfad108908f4e4ea30cf58e370c - actions-runner-osx-arm64-2.314.0.tar.gz 4a24368657a5a994e8c4d2c0e4b304005e6b093ebd3081dd03e5e9436ba52b9e - actions-runner-linux-x64-2.314.0.tar.gz 82ac833f291f0f6b9b546f501043cab1ddb01935281403f435e78ee81a447756 - actions-runner-linux-arm64-2.314.0.tar.gz cc9d28a9e7b6e9e37dea74ce55f5c314724e7f81225b8e3b0d3eaf1f27a90171 - actions-runner-linux-arm-2.314.0.tar.gz a90fbc2d0bcce91be2d3d112cffde0c363b33ef9a807c05f238bcd5b75fa26a7 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Co-authored-by: lumiere-bot[bot] <98047013+lumiere-bot[bot]@users.noreply.github.com>

…14.1 ) (#4287) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/onedr0p/actions-runner](https://ghcr.io/onedr0p/actions-runner) ([source](https://github.com/actions/runner)) | patch | `2.314.0` -> `2.314.1` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>actions/runner (ghcr.io/onedr0p/actions-runner)</summary> ### [`v2.314.1`](https://github.com/actions/runner/releases/tag/v2.314.1) [Compare Source](https://github.com/actions/runner/compare/v2.314.0...v2.314.1) ##### What's Changed - Prepare v2.313.0 Release by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3137 - Pass RunnerOS during job acquire. by [@TingluoHuang](https://github.com/TingluoHuang) in [actions/runner#3140 - Process `snapshot` tokens by [@davidomid](https://github.com/davidomid) in [actions/runner#3135 - Update dotnet sdk to latest version [@6](https://github.com/6).0.419 by [@github-actions](https://github.com/github-actions) in [actions/runner#3158 - handle broker run service exception handling by [@yaananth](https://github.com/yaananth) in [actions/runner#3163 - Add a retry logic to docker login operation by [@enescakir](https://github.com/enescakir) in [actions/runner#3089 - Broker fixes for token refreshes and AccessDeniedException by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3161 - Remove USE_BROKER_FLOW by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3162 - Refresh Token for BrokerServer by [@luketomlinson](https://github.com/luketomlinson) in [actions/runner#3167 - Better step timeout message. by [@TingluoHuang](https://github.com/TingluoHuang) in [actions/runner#3166 - Fix summaries for actions results by [@SrRyan](https://github.com/SrRyan) in [actions/runner#3174 ##### New Contributors - [@davidomid](https://github.com/davidomid) made their first contribution in [actions/runner#3135 - [@enescakir](https://github.com/enescakir) made their first contribution in [actions/runner#3089 - [@SrRyan](https://github.com/SrRyan) made their first contribution in [actions/runner#3174 **Full Changelog**: actions/runner@v2.313.0...v2.314.0 *Note: Actions Runner follows a progressive release policy, so the latest release might not be available to your enterprise, organization, or repository yet. To confirm which version of the Actions Runner you should expect, please view the download instructions for your enterprise, organization, or repository. See https://docs.github.com/en/enterprise-cloud@latest/actions/hosting-your-own-runners/adding-self-hosted-runners* ##### Windows x64 We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows. The following snipped needs to be run on `powershell`: ```powershell ##### Create a folder under the drive root mkdir \actions-runner ; cd \actions-runner ##### Download the latest runner package Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-win-x64-2.314.1.zip -OutFile actions-runner-win-x64-2.314.1.zip ##### Extract the installer Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-x64-2.314.1.zip", "$PWD") ``` ##### \[Pre-release] Windows arm64 **Warning:** Windows arm64 runners are currently in preview status and use [unofficial versions of nodejs](https://unofficial-builds.nodejs.org/). They are not intended for production workflows. We recommend configuring the runner in a root folder of the Windows drive (e.g. "C:\actions-runner"). This will help avoid issues related to service identity folder permissions and long file path restrictions on Windows. The following snipped needs to be run on `powershell`: ```powershell ##### Create a folder under the drive root mkdir \actions-runner ; cd \actions-runner ##### Download the latest runner package Invoke-WebRequest -Uri https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-win-arm64-2.314.1.zip -OutFile actions-runner-win-arm64-2.314.1.zip ##### Extract the installer Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$PWD\actions-runner-win-arm64-2.314.1.zip", "$PWD") ``` ##### OSX x64 ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-osx-x64-2.314.1.tar.gz ##### Extract the installer tar xzf ./actions-runner-osx-x64-2.314.1.tar.gz ``` ##### OSX arm64 (Apple silicon) ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-osx-arm64-2.314.1.tar.gz ##### Extract the installer tar xzf ./actions-runner-osx-arm64-2.314.1.tar.gz ``` ##### Linux x64 ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-linux-x64-2.314.1.tar.gz ##### Extract the installer tar xzf ./actions-runner-linux-x64-2.314.1.tar.gz ``` ##### Linux arm64 ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-linux-arm64-2.314.1.tar.gz ##### Extract the installer tar xzf ./actions-runner-linux-arm64-2.314.1.tar.gz ``` ##### Linux arm ```bash ##### Create a folder mkdir actions-runner && cd actions-runner ##### Download the latest runner package curl -O -L https://github.com/actions/runner/releases/download/v2.314.1/actions-runner-linux-arm-2.314.1.tar.gz ##### Extract the installer tar xzf ./actions-runner-linux-arm-2.314.1.tar.gz ``` ##### Using your self hosted runner For additional details about configuring, running, or shutting down the runner please check out our [product docs.](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/adding-self-hosted-runners) ##### SHA-256 Checksums The SHA-256 checksums for the packages included in this build are shown below: - actions-runner-win-x64-2.314.1.zip 2e1d73c3fe00ad37c359e4f48bd09aa88ef09a46fca16d6f1e94776ccf67fc27 - actions-runner-win-arm64-2.314.1.zip acc807696d1dcad6fb45f6038f884185c54c48127445c365e86d03adb164a9e2 - actions-runner-osx-x64-2.314.1.tar.gz 3faff4667d6d12c41da962580168415d628e3ffba9924b9ac995752087efc921 - actions-runner-osx-arm64-2.314.1.tar.gz e34dab0b4707ad9a9db75f5edf47a804e293af853967a5e0e3b29c8c65f3a004 - actions-runner-linux-x64-2.314.1.tar.gz 6c726a118bbe02cd32e222f890e1e476567bf299353a96886ba75b423c1137b5 - actions-runner-linux-arm64-2.314.1.tar.gz 3d27b1340086115a118e28628a11ae727ecc6b857430c4b1b6cbe64f1f3b6789 - actions-runner-linux-arm-2.314.1.tar.gz a653dd46dafd47c9a3a6637a18161a1445ac6b9c3f6d6b0305be9e1ee65769af </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Co-authored-by: lumiere-bot[bot] <98047013+lumiere-bot[bot]@users.noreply.github.com>

davidomid added 24 commits February 7, 2024 15:24

Added Snapshot TemplateToken to AgentJobRequestMessage

50f18b4

WIP for processing the snapshot token

f760496

Changed snapshot post job step condition to Success, added comments

63636f9

Refactored snapshot post-job step

5c7313e

Added evaluation of snapshot token to retrieve image name

3689161

Added snapshot to workflow schema

b0488ff

Fixed linter error

da26c9d

Merge branch 'main' into users/davidomid/process-snapshot-token

0ef106b

Migrated snapshot logic to new SnapshotOperationProvider

74cad10

Merge branch 'users/davidomid/process-snapshot-token' of https://gith…

2deea1a

…ub.com/davidomid/runner into users/davidomid/process-snapshot-token

Fixed linter error

3d1313c

Fixed linter errors

88a17ab

Fixed linter error

79c17b4

Fixed linter errors

dd926d2

Updated L0 tests

5416b11

Fixed linter errors

aa32d3c

Added new JobExtensionL0 tests for snapshot post-job step

0892678

Added JobExtensionL0 test case for snapshot mappings

f8495a5

Merge branch 'main' into users/davidomid/process-snapshot-token

4ce5b6a

Added SnapshotOperationProviderL0 tests

ca0c01a

Merge branch 'users/davidomid/process-snapshot-token' of https://gith…

e40bbe4

…ub.com/davidomid/runner into users/davidomid/process-snapshot-token

Enabled nullable types for SnapshotOperationProvider and its tests

c855632

Added more assertions to SnapshotOperationProviderL0 tests

5bdadfc

Fixed linter errors

fbc55d2

davidomid changed the title ~~[DRAFT] [WIP] Users/davidomid/process snapshot token~~ Process snapshot token Feb 12, 2024

davidomid marked this pull request as ready for review February 12, 2024 11:58

davidomid requested a review from a team as a code owner February 12, 2024 11:58

Made sure TestHostContexts are disposed of properlyh in SnapshotOpera…

0400051

…tionProviderL0 tests

davidomid changed the title ~~Process snapshot token~~ Process snapshot tokens Feb 12, 2024

ericsciple reviewed Feb 12, 2024

View reviewed changes

src/Runner.Worker/JobExtension.cs Show resolved Hide resolved

ericsciple reviewed Feb 12, 2024

View reviewed changes

src/Runner.Worker/JobExtension.cs Outdated Show resolved Hide resolved

ericsciple previously approved these changes Feb 12, 2024

View reviewed changes