-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding a config_automation.yml #435
Closed
Closed
Changes from 48 commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
3a056ae
Neatening up and trying to control gha triggers
cansavvy 9d154da
Add some config mechs
cansavvy 964e16d
One strategy for yamls
cansavvy bdf4d65
Fix trigger syntax
cansavvy a853cf7
Try to fix syntax part 2
cansavvy 0d84ced
Fix syntax part 3
cansavvy 43266db
Add TRIGGER-START
cansavvy 996e502
This is mostly working
cansavvy a85d3fd
Set up gha
cansavvy 579c801
This seems to work
cansavvy a56d5ba
Updates to script
cansavvy 6d6ca3b
Updates to script 2
cansavvy 975148f
Nope
cansavvy 35e1d60
Don't delete last line
cansavvy a0d2c02
script edit
cansavvy 6d36644
Added some extra lines
cansavvy b200741
Don't add extra spaces
cansavvy e3e658c
Get rid of packages.bib
cansavvy a2f97a6
No packages.bib
cansavvy fa7b618
Add OTTR Website
cansavvy a398c81
Update sync
cansavvy cc6c6e4
Get rid of git_repo_check.R too
cansavvy 9c1382c
Make sure ulrs are running on pull request
cansavvy ccee843
Update all
cansavvy 01cba8a
Fix manage-gha
cansavvy c192ddd
Switch what image we use
cansavvy f69702a
Put extract indices after the file is found
cansavvy ce90b94
If push fails echo
cansavvy 4ecf644
Add docker tester to the file of what's deleted
jhudsl-robot 4433b36
Make render coursera its own thing
jhudsl-robot aae872e
Add blank lines
jhudsl-robot f7cf9c8
Merge remote-tracking branch 'origin/staging' into cansavvy/main-gha
jhudsl-robot ef0d1e6
Add render-coursera.yml to sync.yml
jhudsl-robot 955b491
Polishing
jhudsl-robot 60e27ae
Polishing up comments and etc
jhudsl-robot 28fa844
No spaces
jhudsl-robot 6c31f07
Update trigger handling
cansavvy 1274e9e
Fix GHA branch handling
cansavvy 5a429cc
Echo if no changes to commit
cansavvy 28c0098
Only add github/workflows folder
cansavvy c25a044
force push
cansavvy 2fbb6b7
Try to fix manage GHA trigger
cansavvy 62afbf7
Fix commenting
cansavvy 70cd861
Merge branch 'staging' into cansavvy/main-gha
cansavvy 97eee77
Fix file changes
cansavvy 01fab8e
Merge remote-tracking branch 'origin/cansavvy/main-gha' into cansavvy…
cansavvy 96ad996
Merge branch 'staging' into cansavvy/main-gha
cansavvy 559d556
Merge branch 'staging' into cansavvy/main-gha
cansavvy 9e2d930
Add a gha-triggers.yml file
cansavvy cba55f8
Update workflow name
cansavvy 316072d
Move yml and fix script
cansavvy 0e55b55
Don't need store-triggers.R anymore!
cansavvy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# Candace Savonen Feb 2022 | ||
|
||
name: Update Github Actions Switches | ||
|
||
on: | ||
push: | ||
paths: | ||
- 'config_automation.yml' | ||
|
||
jobs: | ||
update-triggers: | ||
runs-on: ubuntu-latest | ||
container: | ||
image: jhudsl/course_template | ||
|
||
steps: | ||
- name: checkout repo | ||
uses: actions/checkout@v2 | ||
|
||
- name: Login as jhudsl-robot | ||
run: | | ||
git config --local user.email "itcrtrainingnetwork@gmail.com" | ||
git config --local user.name "jhudsl-robot" | ||
|
||
- name: Run GHA switches script | ||
run: Rscript --vanilla scripts/employ-gha-switches.R | ||
|
||
- name: Commit updated workflows to current branch | ||
id: commit_it | ||
run: | | ||
branch_name=$(git rev-parse --abbrev-ref HEAD) | ||
git add .github/workflows --force | ||
git commit -m 'Update GHA switches' || echo "No changes to commit" | ||
git push origin || echo "No changes to commit" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# PR Checks | ||
check-quizzes: yes | ||
url-checker: yes | ||
render-preview: yes | ||
style-and-sp-check: yes | ||
|
||
# Rendering | ||
render-bookdown: yes | ||
render-leanpub: yes | ||
render-coursera: yes | ||
|
||
# Docker specific | ||
docker-build-test: yes |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,77 @@ | ||
#!/usr/bin/env Rscript | ||
|
||
library(magrittr) | ||
|
||
# Make extract trigger function | ||
extract_trigger <- function(gha_contents) { | ||
# Extract trigger criteria | ||
trigger_start <- grep("TRIGGER-START", gha_contents) | ||
trigger_end <- grep("TRIGGER-END", gha_contents) | ||
|
||
return(trigger_start:trigger_end) | ||
} | ||
|
||
# Find .git root directory | ||
root_dir <- rprojroot::find_root(rprojroot::has_dir(".github")) | ||
|
||
### Load in "on" triggers | ||
|
||
# Each github action has its own | ||
on_triggers <- readr::read_rds(file.path(root_dir, ".github", "on-triggers.rds")) | ||
|
||
# Same off trigger for all | ||
off_trigger <- c("#---TRIGGER-START---#", | ||
"on:", | ||
" workflow_dispatch:", | ||
"#---TRIGGER-END----#") | ||
|
||
### Read in and set up config automation yaml | ||
|
||
# Get github actions directory | ||
github_actions_dir <- file.path(root_dir, ".github", "workflows") | ||
|
||
# Get list of github actions files | ||
github_actions_files <- list.files(github_actions_dir, pattern = "\\.yml$", | ||
full.names = TRUE) | ||
|
||
# Set up the list of the gha | ||
config_yaml <- yaml::read_yaml(file.path(root_dir, "config_automation.yml")) %>% | ||
data.frame() %>% | ||
t() %>% | ||
data.frame() %>% | ||
tibble::rownames_to_column("gha_files") %>% | ||
dplyr::mutate(gha_files = paste0(gsub("\\.", "-", gha_files), ".yml")) | ||
|
||
colnames(config_yaml)[2] <- "on_or_off" | ||
|
||
# Read in all files | ||
all_gha <- lapply(github_actions_files, function(gha_file) { | ||
|
||
# Read in yaml | ||
yaml_contents <- readLines(gha_file) | ||
|
||
# Should this be on or off? | ||
status <- config_yaml %>% | ||
dplyr::filter(basename(gha_file) == gha_files) %>% | ||
dplyr::pull("on_or_off") | ||
|
||
if (length(status) == 1) { | ||
message(paste("Turning trigger on:", status, "for", basename(gha_file))) | ||
# Get trigger indices | ||
trigger_indices <- extract_trigger(yaml_contents) | ||
|
||
if (status) { | ||
trigger <- on_triggers[[basename(gha_file)]] | ||
} else { | ||
trigger <- off_trigger | ||
} | ||
|
||
# Remove current trigger | ||
yaml_contents <- yaml_contents[-trigger_indices] | ||
|
||
# Put new trigger in | ||
yaml_contents <- append(yaml_contents, trigger, after = trigger_indices[1] - 1) | ||
|
||
writeLines(yaml_contents, gha_file) | ||
} | ||
}) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Write on-trigger yaml | ||
# This script stores what the 'on' triggers look like for each github action | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the bad script that stores whatever is in the current version of the gha which requires you to turn all the github actions ON before running this. |
||
|
||
# Find .git root directory | ||
root_dir <- rprojroot::find_root(rprojroot::has_dir(".github")) | ||
|
||
# Get github actions directory | ||
github_actions_dir <- file.path(root_dir, ".github", "workflows") | ||
|
||
# Get list of github actions files | ||
github_actions_files <- list.files(github_actions_dir, pattern = "\\.yml$", | ||
full.names = TRUE) | ||
|
||
# Don't manage the manager | ||
github_actions_files <- grep("manage-gha.yml", github_actions_files, value = TRUE, invert = TRUE) | ||
|
||
# Read in all files | ||
all_gha <- lapply(github_actions_files, readLines) | ||
names(all_gha) <- basename(github_actions_files) | ||
|
||
# Make extract trigger function | ||
extract_trigger <- function(gha_contents) { | ||
# Extract trigger criteria | ||
trigger_start <- grep("TRIGGER-START", gha_contents) | ||
trigger_end <- grep("TRIGGER-END", gha_contents) | ||
|
||
return(gha_contents[trigger_start:trigger_end]) | ||
} | ||
|
||
on_triggers <- lapply(all_gha, extract_trigger) | ||
|
||
readr::write_rds(on_triggers, file.path(root_dir, ".github", "on-triggers.rds")) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's also bad that the trigger gets overwritten whether or not there is a change specified in
config_automation.yml
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think my next idea for this, and for the other problem is what if we store a yaml of all the triggers. (
.github/triggers.yml
)So for example:
This format would be far easier to manage than an RDS because we can 1) Just change the triggers in this file directly using a text editor and 2) We can use it to check the current status with the previous status so we can avoid overwriting everytime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My hope is that this kind of yaml would allow the
store-triggers.R
script render unnecessary completely and would make everything a lot more streamlined.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
with the latest iteration of this,
store-triggers.R
is not obsolete and now you just need to specify things in thegha-triggers.yml
if you want to change the triggers.