Prebuilt runner image using packer #1444

ScottGuymer · 2021-11-25T13:22:36Z

When looking at using ephemeral runners it becomes clear that startup time for the agents becomes a key factor in build times.

To enable us to shorten the startup times as much as possible we should build an image that has all the dependencies and the runner installed. In this way there is much less work to do when starting the runner.

To do this I have used packer to build an image that can be used for the runners.

This requires a few changes to how the current scripting is done. We need to refactor this into two phases

installation
startup

Installation
This is all the work that can be done before the runner starts, so installing dependencies or configuring generic settings

Startup
These are the things that can only happen when you are booting an instance of an agent. Such as collecting the GH token to connect the runner or configuring CloudWatch.

So we can pass the runner version in at packer build time if we want to update the runner version.

Retrieve the required config via the instance tags so we dont have to pass in and set environment on the instance in an awkward way.

Give the instance the permission to query its own tags and set the correct tags on the instance.

ScottGuymer · 2021-11-27T15:03:13Z

So a few things i have started to do here

build the image using packer with the current amzn2 image as a base
refactored the install script to pull the runner directly from a URL (removing the need for the s3 bucket at runtime)
Added a startup.sh script to /var/lib/cloud/scripts/per-boot/ so it will run at each boot. This does the following
- Gets the instance meta data required (but now uses tags for some non secret info like environment)
- Gets the GH API token from SSM
- Optionally configures cloudwatch based on tag setting
- configures the runner
- starts the runner and waits for it to exit (this could be made switchable to make it a long lived runner)
- Terminates the instance when the runner exists (mostly for ephemeral workloads)
Given the instance rights to read tags in IAM to be able to pass config in an simple and extensible way.
Added a simple CI validation job to check the format and validate

Whats left to decide/do?

How many/where are we happy with breaking changes to create simplicity?
- Try to keep both ways of working possible
- merge scripts to make them shared across user_data and image provisioning
Do we allow both image based and startup script based config?
- yes allow both
Do we build this image in CI?
- We can try and build to confirm the build is working but not share it just yet
Do we make this ami public and the default image to use instead of the on demand config?
- not yet as we will enable both options for the short term
How do we deal with the pre_install hook?
- this kind of functionality is needed for instance to update resolv.conf or other environment specific stuff
Do we add the post_install hook to the startup.sh so it can run something before the agent starts?
- This would be useful to have as an option
Do we use user_data as a point of customisation for the image on start?
- Yes this should remain in place and be used
How do we handle the arm_patch functionality, should this just be bake into arm images?
- can be done via user_data or similar (or a specific custom image)

refactored to make sure user_data continues to work with minimal breaking changes. Use a single set of scripts shared between image and user_data provisioning.

ScottGuymer · 2021-11-30T17:19:31Z

In testing the prebuilt image is starting in somewhere between 20-25 secs..

Starting runner after 0:0:21

Its best practice to use SSM parameters for configuration of the runners. In adding this i have also added parameter path based config so its easy to extend in the future.

main.tf

modules/runners/logging.tf

modules/runners/main.tf

modules/runners/policies-runner.tf

Keep within existing naming convention

npalm

Tested, all good!

## [0.26.0](v0.25.2...v0.26.0) (2021-12-03) ### Features * Add hooks for prebuilt images (AMI), including amazon linux packer example ([#1444](#1444)) ([060daac](060daac)) ### Bug Fixes * add runners binaries bucket as terraform output ([5809fee](5809fee))

…ker example (#1444) * Initial creation of runner image * Refactored startup script and added it to the per-boot folder * Make the runner location a variable So we can pass the runner version in at packer build time if we want to update the runner version. * Retrieve external config setting via tags Retrieve the required config via the instance tags so we dont have to pass in and set environment on the instance in an awkward way. * Enable tag based config Give the instance the permission to query its own tags and set the correct tags on the instance. * Add a CI job * Fix the CI build * Fix the formatting * Retain user_data provisioning and remove duplication refactored to make sure user_data continues to work with minimal breaking changes. Use a single set of scripts shared between image and user_data provisioning. * Fix interpolation issues in template file * fix build * Fix formatting * minor tweaks and fixes * Fixes from testing * Enable docker on boot * Add in output of start time for the runner * Scoop up the runner log * Add a powershell build script for windows users * Fix formatting * Use SSM parameters for configuration Its best practice to use SSM parameters for configuration of the runners. In adding this i have also added parameter path based config so its easy to extend in the future. * Make the SSM policy more specific * Update .github/workflows/packer-build.yml Co-authored-by: Niek Palm <npalm@users.noreply.github.com> * Added condition to the describe tags policy * Dont use templatefile on the tags policy Because of the use of ${} in the policy terraform is trying to replace it. * Added an option to turn off userdata scripting * Added/updated documentation * Revert policy as it has no effect on the permissions * Add reference to prebuilt images in the main readme * Add an example of deploying with prebuilt images * Update readme * Use current user as ami_owner * Update example to 5 secs * Updated ami name to include the arch * Fixed log file variable * Added explicit info about required settings to the readme * Change userdata_enabled to enabled_userdata Keep within existing naming convention Co-authored-by: Niek Palm <npalm@users.noreply.github.com>

* add option ephemeral runners * fix tests * Add retry mechanisme for scaling errors * Add retry mechanisme for scaling errors Add retry mechanisme for scaling errors Add retry mechanisme for scaling errors Add retry mechanisme for scaling errors * Add tests for lamda handler * Add basic test for ephemeral case * Add basic test for scale down in lambda wrapper * Ensure check_runs are ignored for ephemeral runners * limit termination to only the instance itself * fix: add logging context to runner lambda (#1399) * fix(logging): Add context to scale logs Signed-off-by: Nathaniel McAuliffe <nmcauliffe@expediagroup.com> * Remove testing Signed-off-by: Nathaniel McAuliffe <nmcauliffe@expediagroup.com> * Remove unnecessary import Signed-off-by: Nathaniel McAuliffe <nmcauliffe@expediagroup.com> * Moving log fields to end, adjusting format * feat: Add hooks for prebuilt images (AMI), including amazon linux packer example (#1444) * Initial creation of runner image * Refactored startup script and added it to the per-boot folder * Make the runner location a variable So we can pass the runner version in at packer build time if we want to update the runner version. * Retrieve external config setting via tags Retrieve the required config via the instance tags so we dont have to pass in and set environment on the instance in an awkward way. * Enable tag based config Give the instance the permission to query its own tags and set the correct tags on the instance. * Add a CI job * Fix the CI build * Fix the formatting * Retain user_data provisioning and remove duplication refactored to make sure user_data continues to work with minimal breaking changes. Use a single set of scripts shared between image and user_data provisioning. * Fix interpolation issues in template file * fix build * Fix formatting * minor tweaks and fixes * Fixes from testing * Enable docker on boot * Add in output of start time for the runner * Scoop up the runner log * Add a powershell build script for windows users * Fix formatting * Use SSM parameters for configuration Its best practice to use SSM parameters for configuration of the runners. In adding this i have also added parameter path based config so its easy to extend in the future. * Make the SSM policy more specific * Update .github/workflows/packer-build.yml Co-authored-by: Niek Palm <npalm@users.noreply.github.com> * Added condition to the describe tags policy * Dont use templatefile on the tags policy Because of the use of ${} in the policy terraform is trying to replace it. * Added an option to turn off userdata scripting * Added/updated documentation * Revert policy as it has no effect on the permissions * Add reference to prebuilt images in the main readme * Add an example of deploying with prebuilt images * Update readme * Use current user as ami_owner * Update example to 5 secs * Updated ami name to include the arch * Fixed log file variable * Added explicit info about required settings to the readme * Change userdata_enabled to enabled_userdata Keep within existing naming convention Co-authored-by: Niek Palm <npalm@users.noreply.github.com> * add option ephemeral runners * Add retry mechanisme for scaling errors Add retry mechanisme for scaling errors Add retry mechanisme for scaling errors Add retry mechanisme for scaling errors * add dead letter queue, and refactor * cleanup * cleanup * sync develop * review fix Co-authored-by: Scott Guymer <scott@scottguymer.co.uk> * review fix Co-authored-by: Scott Guymer <scott@scottguymer.co.uk> * review vfix Co-authored-by: Scott Guymer <scott@scottguymer.co.uk> * review vfix Co-authored-by: Scott Guymer <scott@scottguymer.co.uk> * fix review * process review comments * process review comments * review comment * process review comments * Update examples/ephemeral/README.md Co-authored-by: Nathaniel McAuliffe <nmcauliffe@expediagroup.com> * Process review comments * Add docs * review comments * update docs Co-authored-by: Scott Guymer <scott@scottguymer.co.uk> Co-authored-by: Nathaniel McAuliffe <nmcauliffe@expediagroup.com>

## [0.28.0](v0.27.2...v0.28.0) (2022-01-06) ### Features * add option ephemeral runners ([#1374](#1374)) ([2f323d6](2f323d6)), closes [#1399](#1399) [#1444](#1444) * Change default location of runner to `/opt` and fix Ubuntu example ([#1572](#1572)) ([77f350b](77f350b)) * Replace run instance API by create fleet API ([#1556](#1556)) ([27e974d](27e974d)) * Support t4g Graviton instance type ([#1561](#1561)) ([3fa5896](3fa5896)) ### Bug Fixes * Add config for windows ami ([#1525](#1525)) ([7907984](7907984))

* chore(release): 0.17.0 [skip ci] * Adding support for new workflow_job event. ([#1019](#1019)) ([a74e10b](a74e10b)) * chore(release): 0.18.0 [skip ci] * add format checking for lambdas in CI ([#899](#899)) ([#1080](#1080)) ([ae9c277](ae9c277)) * add option to overwrite / disable egress [#748](#748) ([#1112](#1112)) ([9c2548d](9c2548d)) * replace depcrated 'request' dependency by 'node-fetch' ([#903](#903)) ([#1082](#1082)) ([fb51756](fb51756)) * chore(release): 0.18.1 [skip ci] * webhook labels for `workflow_job` ([#1133](#1133)) ([4b39fb9](4b39fb9)) * chore(release): 0.19.0 [skip ci] * **scale-down:** Update Owner Logic ([#1065](#1065)) ([ba2536b](ba2536b)), closes [#2](#2) * explicit set region for downloading runner distribution from S3 ([#1204](#1204)) ([439fb1b](439fb1b)) * upgrade jest ([#1219](#1219)) ([c8b8139](c8b8139)) * use dynamic block to ignore null market opts ([#1202](#1202)) ([df9bd78](df9bd78)) * use dynamic block to ignore null market opts ([#1202](#1202)) ([06a5598](06a5598)) * **logging:** Additional Logging ([#1135](#1135)) ([f7f194d](f7f194d)) * **scale-down:** Clearing cache between runs ([#1164](#1164)) ([e72227b](e72227b)) * chore(release): 0.19.1 [skip ci] * `instance_types` from a Set to a List, so instance order preference is preserved ([#1154](#1154)) ([150d227](150d227)) * chore(release): 0.20.0 [skip ci] * Add option to disable SSL verification support for GitHub Enterprise Server ([#1216](#1216)) ([3c3ef19](3c3ef19)), closes [#1207](#1207) * chore(release): 0.20.1 [skip ci] * Upgrade lambda runtime to node 14.x ([#1203](#1203)) ([570949a](570949a)) * **webhook:** remove node fetch ([ca14ac5](ca14ac5)) * **webhook:** replace node-fetch by axios [#1247](#1247) ([80fff4b](80fff4b)) * added more detailed logging for scaling up and down ([#1222](#1222)) ([9aa7456](9aa7456)) * chore(release): 0.21.0 [skip ci] * Ignore github managed labels and add check disable option ([#1244](#1244)) ([859fa38](859fa38)) * remove unused app client since SSH key is used to secure app authorization ([#1223](#1223)) ([4cb5cf1](4cb5cf1)) * upgrade Terraform version of module 1.0.x ([#1254](#1254)) ([2a817dc](2a817dc)) * chore(release): 0.21.1 [skip ci] * **logging:** Adjusting scale logging messages and levels ([#1286](#1286)) ([665e1a6](665e1a6)) * **logging:** Adjusting webhook logs and levels ([#1287](#1287)) ([9df5fb8](9df5fb8)) * Update launch template to use metadata service v2 ([#1278](#1278)) ([ef16287](ef16287)) * chore(release): 0.22.0 [skip ci] * adding message retention seconds ([#1354](#1354)) ([a19929f](a19929f)) * adding var for tags for ec2s ([#1357](#1357)) ([31cf02d](31cf02d)) * add validation to distribution_bucket_name variable ([#1356](#1356)) ([6522317](6522317)) * chore(release): 0.23.0 [skip ci] * add option to format logging in JSON for lambdas ([#1228](#1228)) ([a250b96](a250b96)) * add option to specify SSE config for dist bucket ([#1324](#1324)) ([ae84302](ae84302)) * reducing verbosity of role and profile ([#1358](#1358)) ([922ef99](922ef99)) * chore(release): 0.23.1 [skip ci] * configurable metadata options for runners ([#1377](#1377)) ([f37df23](f37df23)) * chore(release): 0.24.0 [skip ci] * support single line for app private key ([#1368](#1368)) ([14183ac](14183ac)) * update return codes, no error code for job that are ignored ([#1381](#1381)) ([f9f705f](f9f705f)) * chore(release): 0.25.0 [skip ci] * Add option to configure concurrent running scale up lambda ([#1415](#1415)) ([23ee630](23ee630)) * clean up non used variables in examples ([#1416](#1416)) ([fe65a5f](fe65a5f)) * chore(release): 0.25.1 [skip ci] * Add required providers to module ssm ([#1423](#1423)) ([5b68b7b](5b68b7b)) * chore(release): 0.25.2 [skip ci] * add logging context to runner lambda ([#1399](#1399)) ([0ba0930](0ba0930)) * **logging:** Add context to webhook logs ([#1401](#1401)) ([8094576](8094576)) * chore(release): 0.26.0 [skip ci] * Add hooks for prebuilt images (AMI), including amazon linux packer example ([#1444](#1444)) ([060daac](060daac)) * add runners binaries bucket as terraform output ([5809fee](5809fee)) * chore(release): 0.26.1 [skip ci] * Download lambda ([#1480](#1480)) ([f1b99d9](f1b99d9)) * **syncer:** Add tests, coverage report, and refactor lambda / naming ([#1478](#1478)) ([8266442](8266442)) * install_config_runner -> install_runner ([#1479](#1479)) ([de5b93f](de5b93f)) * chore(release): 0.27.0 [skip ci] * add windows support ([#1476](#1476)) ([dbba705](dbba705)) * chore(release): 0.27.1 [skip ci] * add --preserve-env to start-runner.sh to enable RUNNER_ALLOW_RUNASROOT ([#1537](#1537)) ([1cd9cd3](1cd9cd3)) * remove export from install script. ([#1538](#1538)) ([d32ca1b](d32ca1b)) * chore(release): 0.27.2 [skip ci] * Dowload lambda see [#1541](#1541) for details. ([#1542](#1542)) ([7cb73c8](7cb73c8)) * chore(release): 0.28.0 [skip ci] * add option ephemeral runners ([#1374](#1374)) ([2f323d6](2f323d6)), closes [#1399](#1399) [#1444](#1444) * Change default location of runner to `/opt` and fix Ubuntu example ([#1572](#1572)) ([77f350b](77f350b)) * Replace run instance API by create fleet API ([#1556](#1556)) ([27e974d](27e974d)) * Support t4g Graviton instance type ([#1561](#1561)) ([3fa5896](3fa5896)) * Add config for windows ami ([#1525](#1525)) ([7907984](7907984)) * chore(release): 0.29.0 [skip ci] * Strict label check and replace disable_check_wokflow_job_labels by opt in enable_workflow_job_labels_check ([#1591](#1591)) ([405b11d](405b11d)) * chore(release): 0.30.0 [skip ci] * Add scheduled / pull based scaling for org level runners ([#1577](#1577)) ([8197432](8197432)) * chore(release): 0.30.1 [skip ci] * **runnrs:** Pool runners to allow multiple pool_config objects ([#1621](#1621)) ([c9c7c69](c9c7c69)) * chore(release): 0.31.0 [skip ci] * **packer:** add vars and minor clean up ([#1611](#1611)) ([1c897a4](1c897a4)) * **webhook:** depcrated warning on ts-jest mocked ([#1615](#1615)) ([56c1ece](56c1ece)) * chore(release): 0.32.0 [skip ci] * **runner:** Replace patch by install ICU package for ARM runners ([#1624](#1624)) ([74cfa51](74cfa51)) * **images:** use new runner install location ([#1628](#1628)) ([36c1bf5](36c1bf5)) * **packer:** Add missing RUNNER_ARCHITECTURE for amazn-linux2 ([#1647](#1647)) ([ec497a2](ec497a2)) * chore(release): 0.33.0 [skip ci] * **images:** Added ubuntu-focual example packer configuration ([#1644](#1644)) ([997b171](997b171)) * **examples:** Update AMI filter ([#1673](#1673)) ([39c019c](39c019c)) * chore(release): 0.34.0 [skip ci] * Add output image id used in launch template ([#1676](#1676)) ([a49fab4](a49fab4)) * chore(release): 0.34.1 [skip ci] * **syncer:** Fix for windows binaries in action runner syncer ([#1716](#1716)) ([63e0e27](63e0e27)) * chore(release): 0.34.2 [skip ci] * Limit AWS Terraform Provider to 3.* ([#1741](#1741)) ([0cf2b5d](0cf2b5d)) * **runner:** Cannot disable cloudwatch agent ([#1738](#1738)) ([0f798ca](0f798ca)) * chore(release): 0.35.0 [skip ci] * Parameterise delete_on_termination ([#1758](#1758)) ([6282351](6282351)), closes [#1745](#1745) * **runner:** Ability to disable default runner security group creation ([#1718](#1718)) ([94779f8](94779f8)) * chore(release): 0.36.0 [skip ci] * **runner:** Add option to disable auto update ([#1791](#1791)) ([c2a834f](c2a834f)) * chore(release): 0.37.0 [skip ci] * Add associate_public_ip_address variable to windows AMI too ([#1819](#1819)) ([0b8e1fc](0b8e1fc)), closes [/github.com//pull/1816#issuecomment-1060650668](https://github.com/philips-labs//github.com/philips-labs/terraform-aws-github-runner/pull/1816/issues/issuecomment-1060650668) * Add associate_public_ip_address variable ([#1816](#1816)) ([052e9f8](052e9f8)) * Add option for KMS encryption for cloudwatch log groups ([#1833](#1833)) ([3f1a67f](3f1a67f)) * Add SQS queue resource policy to improve security ([#1798](#1798)) ([96def9a](96def9a)) * Add Support for Alternative Partitions in ARNs (like govcloud) ([#1815](#1815)) ([0ba06c8](0ba06c8)) * Add variable to specify custom commands while building the AMI ([#1838](#1838)) ([8f9c342](8f9c342)) * Autoupdate should be disabled by default ([#1797](#1797)) ([828bed6](828bed6)) * Create SQS DLQ policy only if DLQ is created ([#1839](#1839)) ([c88a005](c88a005)) * Upgrade Amazon base AMI to Amazon Linux 2 kernel 5x ([#1812](#1812)) ([9aa5532](9aa5532)) * chore(release): 0.38.0 [skip ci] * Add option for ephemeral to check builds status before scaling ([#1854](#1854)) ([7eb0bda](7eb0bda)) * Retention days was used instead of kms key id for pool ([#1855](#1855)) ([aa29d93](aa29d93)) * chore(release): 0.39.0 [skip ci] * Add possibility to create multiple ebs ([#1845](#1845)) ([7a2ca0d](7a2ca0d)) * Don't delete busy runners ([#1832](#1832)) ([0e9b083](0e9b083)) * chore(release): 0.40.0 [skip ci] * Support multi runner process support for runner scale down. ([#1859](#1859)) ([3658d6a](3658d6a)) * Set the minimal AWS provider to 3.50 ([#1937](#1937)) ([16095d8](16095d8)) * chore(release): 0.40.1 [skip ci] * Avoid non semantic commontes can be merged. ([#1969](#1969)) ([ad1c872](ad1c872)) * chore(release): 0.40.2 [skip ci] * Outputs for pool need to account for complexity ([#1970](#1970)) ([2d92906](2d92906)) * chore(release): 0.40.3 [skip ci] * Volume size is ingored ([#2014](#2014)) ([b733248](b733248)), closes [#1954](#1954) * chore(release): 0.40.4 [skip ci] * Wrong block device mapping ([#2019](#2019)) ([c42a467](c42a467)) * chore(release): 1.0.0 [skip ci] * var.volume_size replaced by var.block_device_mappings * The module is upgraded to AWS Terraform provider 4.x * Improve syncer s3 kms encryption ([38ed5be](38ed5be)) * Remove var.volume_size in favour of var.block_device_mappings ([4e97048](4e97048)) * Support AWS 4.x Terraform provider ([#1739](#1739)) ([cfb6da2](cfb6da2)) * Wrong block device mapping ([#2019](#2019)) ([185ef20](185ef20)) * chore(release): 1.1.0 [skip ci] * Add option to enable detailed monitoring for runner launch template ([#2024](#2024)) ([e73a267](e73a267)) * chore(release): 1.1.1 [skip ci] * **runner:** Don't treat the string "false" as true. ([#2051](#2051)) ([b67c7dc](b67c7dc)) * chore(release): 1.2.0 [skip ci] * Replace environment variable by prefix ([#1858](#1858)) ([e2f9a27](e2f9a27)) * docs: fix hyperlinks in the Terraform Registry documentation (#2085) This makes the hyperlink correct in the Terraform Registry documentation * chore(release): 1.3.0 [skip ci] * Support arm64 lambda functions ([#2121](#2121)) ([9e2a7b6](9e2a7b6)) * Support Node16 for AWS Lambda ([#2073](#2073)) ([68a2014](68a2014)) * replaced old environment variable ([#2146](#2146)) ([f2072f7](f2072f7)) * set explicit permissions on s3 for syncer lambda ([#2145](#2145)) ([aa7edd1](aa7edd1)) * set kms key on aws_s3_object when encryption is enabled ([#2147](#2147)) ([b4dc706](b4dc706)) * chore(release): 1.4.0 [skip ci] * Add option to match some of the labes instead of all [#2122](#2122) ([#2123](#2123)) ([c5e3c21](c5e3c21)) * don't apply extra labels unless defined ([#2181](#2181)) ([c0b11bb](c0b11bb)) * Remove asterik in permission for runner lambda to describe instances ([9b9da03](9b9da03)) * chore(release): 1.4.1 [skip ci] * added server_side_encryption key to download trigger for distribution ([#2207](#2207)) ([404e3b6](404e3b6)) * chore(release): 1.5.0 [skip ci] * Add ubuntu-jammy example image based on existing ubuntu-focal ([#2102](#2102)) ([486ae91](486ae91)) * **images:** avoid wrong AMI could be selected for ubuntu focal ([#2214](#2214)) ([76be94b](76be94b)) * chore(release): 1.6.0 [skip ci] * Add options extra option to ebs block device mapping ([#2052](#2052)) ([7cd2524](7cd2524)) * Enable node16 default ([#2074](#2074)) ([58aa5ed](58aa5ed)) * Incorrect path of Runner logs ([#2233](#2233)) ([98eff98](98eff98)) * Preventing that lambda webhook fails when it tries to process an installation_repositories event ([#2288](#2288)) ([8656c83](8656c83)) * Update ubuntu example to fix /opt/hostedtoolcache ([#2302](#2302)) ([8eea748](8eea748)) * Webhook lambda misleading log ([#2291](#2291)) ([c6275f9](c6275f9)) * chore(release): 1.7.0 [skip ci] * Webhook accept jobs where not all labels are provided in job. ([#2209](#2209)) ([6d9116f](6d9116f)) * Ignore case for runner labels. ([#2315](#2315)) ([014985a](014985a)) * chore(release): 1.8.0 [skip ci] * Add option to disable lambda to sync runner binaries ([#2314](#2314)) ([9f7d32d](9f7d32d)) * **examples:** Upgrading ubuntu example to 22.04 ([#2250](#2250)) ([d4b7650](d4b7650)), closes [#2103](#2103) * chore(release): 1.8.1 [skip ci] * **runners:** Pass allocation strategy ([#2345](#2345)) ([68d3445](68d3445)) * chore(release): 1.9.0 [skip ci] * Add option to enable access log for API gateway ([#2387](#2387)) ([fcd9fba](fcd9fba)) * add s3_location_runner_distribution var as expandable for userdata ([#2371](#2371)) ([05fe737](05fe737)) * Encrypted data at REST on SQS by default ([#2431](#2431)) ([7f3f4bf](7f3f4bf)) * **images:** Allow passing instance type when building windows image ([#2369](#2369)) ([eca23bf](eca23bf)) * **runners:** Fetch instance environment tag though metadata ([#2346](#2346)) ([27db290](27db290)) * **runners:** Set the default Windows AMI to Server 2022 ([#2325](#2325)) ([78e99d1](78e99d1)) * chore(release): 1.9.1 [skip ci] * **webhook:** Use `x-hub-signature-256` header as default ([#2434](#2434)) ([9c3e495](9c3e495)) * chore(release): 1.10.0 [skip ci] * Download runner release via latest release API ([#2455](#2455)) ([e75e092](e75e092)) * fix: Execute runner in own process, mask token in logs * Add option to disable user_data logging * Enforcing debug is disabled, and introduce option to enable debug logging. * add section related to security considerations * add section related to security considerations Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net> Co-authored-by: Derek Crosson <derekcrosson18@gmail.com>

## [1.12.0](v1.11.0...v1.12.0) (2022-10-12) ### Features * Added the AMI to machine setup info to runner workflows. ([#2451](#2451)) ([e197cbd](e197cbd)) * **images:** add ami for windows core 2022 ([#2390](#2390)) ([97707c2](97707c2)) * Log workflow id in webhook ([#2511](#2511)) ([204acf1](204acf1)) * Security improvements, add option to disable userdata logging ([9a9e2ee](9a9e2ee)), closes [#1019](#1019) [#899](#899) [#1080](#1080) [#748](#748) [#1112](#1112) [#903](#903) [#1082](#1082) [#1133](#1133) [#2](#2) [#1204](#1204) [#1219](#1219) [#1202](#1202) [#1202](#1202) [#1135](#1135) [#1164](#1164) [#1154](#1154) [#1207](#1207) [#1203](#1203) [#1247](#1247) [#1222](#1222) [#1244](#1244) [#1223](#1223) [#1254](#1254) [#1286](#1286) [#1287](#1287) [#1278](#1278) [#1354](#1354) [#1357](#1357) [#1356](#1356) [#1228](#1228) [#1324](#1324) [#1358](#1358) [#1377](#1377) [#1368](#1368) [#1381](#1381) [#1415](#1415) [#1416](#1416) [#1423](#1423) [#1399](#1399) [#1401](#1401) [#1444](#1444) [#1480](#1480) [#1478](#1478) [#1479](#1479) [#1476](#1476) [#1537](#1537) [#1538](#1538) [#1541](#1541) [#1542](#1542) [#1399](#1399) [#1444](#1444) [#1572](#1572) [#1556](#1556) [#1561](#1561) [#1525](#1525) [#1591](#1591) [#1577](#1577) [#1621](#1621) [#1611](#1611) [#1615](#1615) [#1624](#1624) [#1628](#1628) [#1647](#1647) [#1644](#1644) [#1673](#1673) [#1676](#1676) [#1716](#1716) [#1741](#1741) [#1738](#1738) [#1745](#1745) [#1718](#1718) [#1791](#1791) [github.com//pull/1816#issuecomment-1060650668](https://github.com/philips-labs/github.com/philips-labs/terraform-aws-github-runner/pull/1816/issues/issuecomment-1060650668) [#1816](#1816) [#1833](#1833) [#1798](#1798) [#1815](#1815) [#1838](#1838) [#1797](#1797) [#1839](#1839) [#1812](#1812) [#1854](#1854) [#1855](#1855) [#1845](#1845) [#1832](#1832) [#1859](#1859) [#1937](#1937) [#1969](#1969) [#1970](#1970) [#1954](#1954) [#2019](#2019) [#1739](#1739) [#2019](#2019) [#2024](#2024) [#2051](#2051) [#1858](#1858) [#2085](#2085) [#2121](#2121) [#2073](#2073) [#2146](#2146) [#2145](#2145) [#2147](#2147) [#2122](#2122) [#2123](#2123) [#2181](#2181) [#2207](#2207) [#2102](#2102) [#2214](#2214) [#2052](#2052) [#2074](#2074) [#2233](#2233) [#2288](#2288) [#2302](#2302) [#2291](#2291) [#2209](#2209) [#2315](#2315) [#2314](#2314) [#2103](#2103) [#2345](#2345) [#2387](#2387) [#2371](#2371) [#2431](#2431) [#2369](#2369) [#2346](#2346) [#2325](#2325) [#2434](#2434) [#2455](#2455) ### Bug Fixes * Remove resource group from module ([#2512](#2512)) ([2628352](2628352))

ScottGuymer added 8 commits November 25, 2021 14:16

Initial creation of runner image

ccf78b3

Refactored startup script and added it to the per-boot folder

563391b

Make the runner location a variable

d8da7a4

So we can pass the runner version in at packer build time if we want to update the runner version.

Retrieve external config setting via tags

9a4bb2f

Retrieve the required config via the instance tags so we dont have to pass in and set environment on the instance in an awkward way.

Enable tag based config

45b4aaa

Give the instance the permission to query its own tags and set the correct tags on the instance.

Add a CI job

17e83d9

Fix the CI build

64a8b59

Fix the formatting

e8b3571

This was referenced Nov 29, 2021

[ephemeral] Add option ephemeral runners #1375

Closed

Donation: Runner AMI template + slimmed-down user_data to match #1313

Closed

options to avoid terminating/spinning up new instances #1166

Closed

ScottGuymer added 11 commits November 29, 2021 18:51

Retain user_data provisioning and remove duplication

9d4e660

refactored to make sure user_data continues to work with minimal breaking changes. Use a single set of scripts shared between image and user_data provisioning.

Fix interpolation issues in template file

f8ad58c

fix build

b3d2fcf

Fix formatting

9e0612e

minor tweaks and fixes

8233ca9

Fixes from testing

4144634

Enable docker on boot

80b1b90

Add in output of start time for the runner

33e9105

Scoop up the runner log

af9198f

Add a powershell build script for windows users

08a0a66

Fix formatting

925ffb9

ScottGuymer marked this pull request as ready for review November 30, 2021 17:20

Use SSM parameters for configuration

11f5a2c

Its best practice to use SSM parameters for configuration of the runners. In adding this i have also added parameter path based config so its easy to extend in the future.

ScottGuymer commented Dec 1, 2021

View reviewed changes

main.tf Show resolved Hide resolved

ScottGuymer commented Dec 1, 2021

View reviewed changes

modules/runners/logging.tf Show resolved Hide resolved

ScottGuymer commented Dec 1, 2021

View reviewed changes

modules/runners/main.tf Show resolved Hide resolved

ScottGuymer commented Dec 1, 2021

View reviewed changes

modules/runners/policies-runner.tf Outdated Show resolved Hide resolved

ScottGuymer added 2 commits December 3, 2021 10:03

Added explicit info about required settings to the readme

49b8a5c

Change userdata_enabled to enabled_userdata

8e987a8

Keep within existing naming convention

npalm approved these changes Dec 3, 2021

View reviewed changes

npalm merged commit 060daac into philips-labs:develop Dec 3, 2021

ScottGuymer deleted the packer_image_build branch December 8, 2021 13:51

ScottGuymer mentioned this pull request Dec 10, 2021

fix: #1218 Proxy implementation for Lambdas (basic UT for aws-sdk v2) #1256

Closed

npalm mentioned this pull request Dec 28, 2021

fix: ubuntu example #1572

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prebuilt runner image using packer #1444

Prebuilt runner image using packer #1444

ScottGuymer commented Nov 25, 2021

ScottGuymer commented Nov 27, 2021 •

edited

Loading

ScottGuymer commented Nov 30, 2021

npalm left a comment

Prebuilt runner image using packer #1444

Prebuilt runner image using packer #1444

Conversation

ScottGuymer commented Nov 25, 2021

ScottGuymer commented Nov 27, 2021 • edited Loading

ScottGuymer commented Nov 30, 2021

npalm left a comment

Choose a reason for hiding this comment

ScottGuymer commented Nov 27, 2021 •

edited

Loading