👔 Restructure automated integration and regression testing flow

[shnizzedy]

Fixes

Resolves #1 by @shnizzedy
Related to FCP-INDI/C-PAC#1211 (comment) by @shnizzedy

Description

This is the main, largest PR for updating the flow for running automated integration and regression tests on bridges-2. I've tried to keep the README in this branch updated as I've iterated through changes to the flow.

Technical details

This PR is an overhaul of the flow, moving from manual launches to full automation.

• Updates GitHub workflows and scripts to move to automation.
  • Adds a workflow to generate and push the results comment via cpac_regsuite_generate_comment
• Packaging
• Linting
• Adds a contributing guide
  • To protect against pwn request attacks, requires the workflows mentioned in the contributing guide to be unchanged in PRs. I committed these workflows directly to the target branch so this PR can also be restricted by these checks.
• Implements the first working iteration of the structure diagrammed in "What this repository does once launched"

graph TB


launch_regtest_lite --> build_image{build_image}
build_image --"success"--> regtest_lite --> status_regtest_lite_success
build_image --"failure"--> status_regtest_lite_failure
regtest_lite --> launch_jobs
subgraph launch_jobs["for PIPELINE in ${PRECONFIGS} for DATA in ${DATA_SOURCE}"]
  reglite{"reglite_${IMAGE_NAME}_${PIPELINE}_${DATA}"}
  gh_initiate_check["Initiate Check\n(GitHub Check for specific run)"]
end


subgraph status_regtest_lite["status_regtest_lite\n(Update GitHub Check status for launch)"]
  status_regtest_lite_success[[success]]
  status_regtest_lite_failure[[failure]]
end


status_regtest_lite_failure --> push_to_github["push_to_github\n(push logs to GitHub)"]


reglite --> FULL_SUCCESS_DEPENDENCIES --> push_to_github
reglite --"success"--> correlate_regtest_lite
reglite --"failure"--> failed_regtest_lite["failed_regtest_lite\n(Update GitHub Check status for specific run)"]


correlate_regtest_lite -. "correlations not connected yet,\nbut will feed into dependencies\nonce they are" .-> FULL_SUCCESS_DEPENDENCIES


subgraph FULL_SUCCESS_DEPENDENCIES["${FULL_SUCCESS_DEPENDENCIES}\n(SLURM job statuses)"]
  
end

Loading

Screenshot

Checklist

• My pull request has a descriptive title (not a vague title like Update index.md).
• My pull request targets the main branch of the repository.
• My commit messages follow best practices.
• My code follows the established code style of the repository.
• I added tests for the changes I made (if applicable).
• I updated the changelog.
• I added or updated documentation (if applicable).
• I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.