Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dynamically provision self-hosted runner #264

Merged
merged 35 commits into from
Sep 12, 2023
Merged

Dynamically provision self-hosted runner #264

merged 35 commits into from
Sep 12, 2023

Conversation

ohinds
Copy link
Contributor

@ohinds ohinds commented Sep 12, 2023

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Summary

Instead of dynamically spinning up an EC2 instance that is statically configured as a github runner, also dynamically register the self-hosted runner just for this run.

Fixes #259

Checklist

  • I have added tests to cover my changes
  • I have updated documentation (if necessary)

Acknowledgment

  • I acknowledge that this contribution will be available under the Apache 2 license.

@ohinds
Add steps to provision and register a github runner, then destroy it
60bbae8
@ohinds
Runs on needs to be first?
baf3e97
@ohinds
Only provide runner name, not self-hosted?
20040ee
@ohinds
scp files individually.
36c935e
@ohinds
Add runner key secret to env
9214847
@ohinds
Can I if always in a step?
726b15f
@ohinds
Key to cwd
0616e1f
@ohinds
Pass hostname to next step
9f4bcb9
@ohinds
Use the right key, and wait for the instance to come up.
573072f
@ohinds
Add a sleep to make sure sshd comes up.
bd293b9
@ohinds
Key perms and quote loop var
7047e34
@ohinds
Sleep more
8c8d3eb
@ohinds
Sleep more than more
a53bd10
@ohinds
Realllly make sure it isn't sshd not coming up
84dc72b
@ohinds
Try new key format
c9b69f7
@ohinds
Try using ssh key github action
8905698
@ohinds
Transfer the correct file.
e62c79a
@ohinds
Clone for scripts and correct destroy
e67bb2a
@ohinds
ssh to a host to execute.
9b04a46
@ohinds
Add self-hosted to runs on
60a8adf
@ohinds
Runs-on order matters?
c2983d3
@ohinds
Multiple runs-on?
ca9c7e7
@ohinds
Export whole runs-on array as a variable
1a3f414
@ohinds
Try a simplier runner id
fca5e09
@ohinds
Sync runner label and ad gh token to destroy
daa9ae7
@ohinds
Hack for now to run on self-hosted
063aa19
@ohinds
Try using existing app to spin up instances
efbe1ea
@ohinds
Correct job name
ae55f43
@ohinds
Default ubuntu python exec
ab490b9
@ohinds
Add aws region as a gh variable
245a70e
@ohinds ohinds added the bug label Sep 12, 2023
@ohinds ohinds merged commit 70a088c into master Sep 12, 2023
9 checks passed
@ohinds ohinds deleted the ohinds-provision-gh-runner branch September 12, 2023 01:20
@hvgazula
Copy link
Contributor

@ohinds The keepalive.yml is no longer needed. Is it?

@ohinds ohinds mentioned this pull request Sep 13, 2023
Merged
6 tasks
ohinds added a commit that referenced this pull request Sep 13, 2023
@ohinds
Remove scripts that shouldn't have gone in with #264.
d70e424
hvgazula pushed a commit that referenced this pull request Sep 13, 2023
@ohinds
Remove unnecessary keepalive runner (#265)
07c14f4 
* Remove unnecessary keepalive runner.

* Remove scripts that shouldn't have gone in with #264.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@satra satra satra approved these changes

@hvgazula hvgazula Awaiting requested review from hvgazula

Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Guide notebook regression should use a dynamically provisioned github self-hosted runner
3 participants
@ohinds @hvgazula @satra