Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a config_automation.yml #435

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
3a056ae
Neatening up and trying to control gha triggers
cansavvy Feb 8, 2022
9d154da
Add some config mechs
cansavvy Feb 8, 2022
964e16d
One strategy for yamls
cansavvy Feb 8, 2022
bdf4d65
Fix trigger syntax
cansavvy Feb 8, 2022
a853cf7
Try to fix syntax part 2
cansavvy Feb 8, 2022
0d84ced
Fix syntax part 3
cansavvy Feb 8, 2022
43266db
Add TRIGGER-START
cansavvy Feb 8, 2022
996e502
This is mostly working
cansavvy Feb 8, 2022
a85d3fd
Set up gha
cansavvy Feb 8, 2022
579c801
This seems to work
cansavvy Feb 8, 2022
a56d5ba
Updates to script
cansavvy Feb 8, 2022
6d6ca3b
Updates to script 2
cansavvy Feb 8, 2022
975148f
Nope
cansavvy Feb 8, 2022
35e1d60
Don't delete last line
cansavvy Feb 8, 2022
a0d2c02
script edit
cansavvy Feb 8, 2022
6d36644
Added some extra lines
cansavvy Feb 8, 2022
b200741
Don't add extra spaces
cansavvy Feb 8, 2022
e3e658c
Get rid of packages.bib
cansavvy Feb 8, 2022
a2f97a6
No packages.bib
cansavvy Feb 8, 2022
fa7b618
Add OTTR Website
cansavvy Feb 8, 2022
a398c81
Update sync
cansavvy Feb 8, 2022
cc6c6e4
Get rid of git_repo_check.R too
cansavvy Feb 8, 2022
9c1382c
Make sure ulrs are running on pull request
cansavvy Feb 8, 2022
ccee843
Update all
cansavvy Feb 8, 2022
01cba8a
Fix manage-gha
cansavvy Feb 8, 2022
c192ddd
Switch what image we use
cansavvy Feb 8, 2022
f69702a
Put extract indices after the file is found
cansavvy Feb 8, 2022
ce90b94
If push fails echo
cansavvy Feb 8, 2022
4ecf644
Add docker tester to the file of what's deleted
jhudsl-robot Feb 8, 2022
4433b36
Make render coursera its own thing
jhudsl-robot Feb 8, 2022
aae872e
Add blank lines
jhudsl-robot Feb 8, 2022
f7cf9c8
Merge remote-tracking branch 'origin/staging' into cansavvy/main-gha
jhudsl-robot Feb 8, 2022
ef0d1e6
Add render-coursera.yml to sync.yml
jhudsl-robot Feb 9, 2022
955b491
Polishing
jhudsl-robot Feb 9, 2022
60e27ae
Polishing up comments and etc
jhudsl-robot Feb 9, 2022
28fa844
No spaces
jhudsl-robot Feb 9, 2022
6c31f07
Update trigger handling
cansavvy Feb 9, 2022
1274e9e
Fix GHA branch handling
cansavvy Feb 9, 2022
5a429cc
Echo if no changes to commit
cansavvy Feb 9, 2022
28c0098
Only add github/workflows folder
cansavvy Feb 9, 2022
c25a044
force push
cansavvy Feb 9, 2022
2fbb6b7
Try to fix manage GHA trigger
cansavvy Feb 9, 2022
62afbf7
Fix commenting
cansavvy Feb 9, 2022
70cd861
Merge branch 'staging' into cansavvy/main-gha
cansavvy Feb 9, 2022
97eee77
Fix file changes
cansavvy Feb 9, 2022
01fab8e
Merge remote-tracking branch 'origin/cansavvy/main-gha' into cansavvy…
cansavvy Feb 9, 2022
96ad996
Merge branch 'staging' into cansavvy/main-gha
cansavvy Feb 9, 2022
559d556
Merge branch 'staging' into cansavvy/main-gha
cansavvy Feb 9, 2022
9e2d930
Add a gha-triggers.yml file
cansavvy Feb 9, 2022
cba55f8
Update workflow name
cansavvy Feb 9, 2022
316072d
Move yml and fix script
cansavvy Feb 9, 2022
0e55b55
Don't need store-triggers.R anymore!
cansavvy Feb 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .github/on-triggers.rds
Binary file not shown.
22 changes: 22 additions & 0 deletions .github/sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,25 @@ group:
dest: scripts/spell-check.R
repos: |
jhudsl/OTTR_Template_Website

repos: |
jhudsl/OTTR_Template_Website
- source: .github/workflows/style-and-sp-check.yml
dest: .github/workflows/style-and-sp-check.yml
- source: .github/workflows/url-checker.yml
dest: .github/workflows/url-checker.yml
- source: scripts/spell-check.R
dest: scripts/spell-check.R
###### TO ADD NEW REPOSITORY USE THIS FORMAT; Feel free to add/drop files that
# you specifically would like synced (or not).
#
# # Repositories to receive changes
# repos: |
# org/repo_name
# # Files which should be updated
# - source: .github/workflows/
# dest: .github/workflows/
# - source: scripts/
# dest: scripts/
# - source: docker/
# dest: docker/
34 changes: 34 additions & 0 deletions .github/workflows/manage-gha.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Candace Savonen Feb 2022

name: Update Github Actions Switches

on:
push:
paths:
- 'config_automation.yml'

jobs:
update-triggers:
runs-on: ubuntu-latest
container:
image: jhudsl/course_template

steps:
- name: checkout repo
uses: actions/checkout@v2

- name: Login as jhudsl-robot
run: |
git config --local user.email "itcrtrainingnetwork@gmail.com"
git config --local user.name "jhudsl-robot"

- name: Run GHA switches script
run: Rscript --vanilla scripts/employ-gha-switches.R

- name: Commit updated workflows to current branch
id: commit_it
run: |
branch_name=$(git rev-parse --abbrev-ref HEAD)
git add .github/workflows --force
git commit -m 'Update GHA switches' || echo "No changes to commit"
git push origin || echo "No changes to commit"
13 changes: 13 additions & 0 deletions config_automation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# PR Checks
check-quizzes: yes
url-checker: yes
render-preview: yes
style-and-sp-check: yes

# Rendering
render-bookdown: yes
render-leanpub: yes
render-coursera: yes

# Docker specific
docker-build-test: yes
77 changes: 77 additions & 0 deletions scripts/employ-gha-switches.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#!/usr/bin/env Rscript

library(magrittr)

# Make extract trigger function
extract_trigger <- function(gha_contents) {
# Extract trigger criteria
trigger_start <- grep("TRIGGER-START", gha_contents)
trigger_end <- grep("TRIGGER-END", gha_contents)

return(trigger_start:trigger_end)
}

# Find .git root directory
root_dir <- rprojroot::find_root(rprojroot::has_dir(".github"))

### Load in "on" triggers

# Each github action has its own
on_triggers <- readr::read_rds(file.path(root_dir, ".github", "on-triggers.rds"))

# Same off trigger for all
off_trigger <- c("#---TRIGGER-START---#",
"on:",
" workflow_dispatch:",
"#---TRIGGER-END----#")

### Read in and set up config automation yaml

# Get github actions directory
github_actions_dir <- file.path(root_dir, ".github", "workflows")

# Get list of github actions files
github_actions_files <- list.files(github_actions_dir, pattern = "\\.yml$",
full.names = TRUE)

# Set up the list of the gha
config_yaml <- yaml::read_yaml(file.path(root_dir, "config_automation.yml")) %>%
data.frame() %>%
t() %>%
data.frame() %>%
tibble::rownames_to_column("gha_files") %>%
dplyr::mutate(gha_files = paste0(gsub("\\.", "-", gha_files), ".yml"))

colnames(config_yaml)[2] <- "on_or_off"

# Read in all files
all_gha <- lapply(github_actions_files, function(gha_file) {

# Read in yaml
yaml_contents <- readLines(gha_file)

# Should this be on or off?
status <- config_yaml %>%
dplyr::filter(basename(gha_file) == gha_files) %>%
dplyr::pull("on_or_off")

if (length(status) == 1) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also bad that the trigger gets overwritten whether or not there is a change specified in config_automation.yml

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my next idea for this, and for the other problem is what if we store a yaml of all the triggers. (.github/triggers.yml)
So for example:

check-quizzes: 
  status: on 
  on: <What the trigger for on looks like>

  off: <What the trigger for off looks like>

render-bookdown: 
  status: on
  on: < ... > 

  off: < ... > 

This format would be far easier to manage than an RDS because we can 1) Just change the triggers in this file directly using a text editor and 2) We can use it to check the current status with the previous status so we can avoid overwriting everytime.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My hope is that this kind of yaml would allow the store-triggers.R script render unnecessary completely and would make everything a lot more streamlined.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the latest iteration of this, store-triggers.R is not obsolete and now you just need to specify things in the gha-triggers.yml if you want to change the triggers.

message(paste("Turning trigger on:", status, "for", basename(gha_file)))
# Get trigger indices
trigger_indices <- extract_trigger(yaml_contents)

if (status) {
trigger <- on_triggers[[basename(gha_file)]]
} else {
trigger <- off_trigger
}

# Remove current trigger
yaml_contents <- yaml_contents[-trigger_indices]

# Put new trigger in
yaml_contents <- append(yaml_contents, trigger, after = trigger_indices[1] - 1)

writeLines(yaml_contents, gha_file)
}
})
32 changes: 32 additions & 0 deletions scripts/store-triggers.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Write on-trigger yaml
# This script stores what the 'on' triggers look like for each github action
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the bad script that stores whatever is in the current version of the gha which requires you to turn all the github actions ON before running this.


# Find .git root directory
root_dir <- rprojroot::find_root(rprojroot::has_dir(".github"))

# Get github actions directory
github_actions_dir <- file.path(root_dir, ".github", "workflows")

# Get list of github actions files
github_actions_files <- list.files(github_actions_dir, pattern = "\\.yml$",
full.names = TRUE)

# Don't manage the manager
github_actions_files <- grep("manage-gha.yml", github_actions_files, value = TRUE, invert = TRUE)

# Read in all files
all_gha <- lapply(github_actions_files, readLines)
names(all_gha) <- basename(github_actions_files)

# Make extract trigger function
extract_trigger <- function(gha_contents) {
# Extract trigger criteria
trigger_start <- grep("TRIGGER-START", gha_contents)
trigger_end <- grep("TRIGGER-END", gha_contents)

return(gha_contents[trigger_start:trigger_end])
}

on_triggers <- lapply(all_gha, extract_trigger)

readr::write_rds(on_triggers, file.path(root_dir, ".github", "on-triggers.rds"))