Clearer descriptions and README

.github/workflows/README.md

@@ -0,0 +1,62 @@
+# Overview
+
+Our CI workflows make heavy usage of [Reusable Workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows). These reusable workflows can then be tested manually via the `manual_dispatch.yml` workflow.
+Or integrated into CI jobs such has `on_pull_request.yml` or `on_main.yml`.
+
+By convention:
+- All reusable workflows start with the `reusable_` prefix.
+- All workflows that are triggered via `workflow_dispatch` start with the `manual_` prefix.
+- All workflows that are triggered via an event start with the `on_` prefix.
+  - `on_pull_request` is triggered on pull requests.
+  - `on_main` is triggered on pushes to the main branch.
+
+## Reusable Workflows
+- [reusable_checks.yml](reusable_checks.yml) - These are all the checks that run to ensure the code is formatted,
+  linted, and tested. This job produces no artifacts other than a pass/fail criteria for the build.
+  - `SAVE_CACHE` - If true, the rust cache will be saved. Generally we only do this for builds on `main`
+- [reusable_bench.yml](reusable_bench.yml) - This job runs the benchmarks to check for performance regressions.
+  - `SAVE_BENCH` - If true, then the benchmark results are saved to update https://ref.rerun.io/dev/bench/
+- [reusable_deploy_docs](reusable_deploy_docs.yml) - This job deploys the python and rust documentation to https://ref.rerun.io
+  - `PY_DOCS_VERSION_NAME` - The name to use for versioning the python docs. This should generally match the version in
+    `Cargo.toml`.
+  - `UPDATE_LATEST` - If true, then the docs will be deployed to `latest/` as well as the versioned directory.
+- [reusable_build_and_test_wheels.yml](reusable_build_and_test_wheels.yml) - This job builds the wheels, runs the
+end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise
+uploaded anywhere.
+  - `MATURIN_FEATURE_FLAGS` - The feature flags to pass to maturin.
+  - `PLATFORM` - Which platform to build for: `linux`, `macos-arm`, `macos-intel`, or `windows`.
+  - `RELEASE_VERSION` - If producing a release, the version number. This must match the version in `Cargo.toml`.
+  - `RRD_ARTIFACT_NAME` - Intermediate name of the GitHub rrd artifact for passing to `reusable_upload_wheels.yml`
+  - `SAVE_CACHE` - If true, the rust cache will be saved. Generally we only do this for builds on `main`
+  - `WHEEL_ARTIFACT_NAME` - Intermediate name of the GitHub wheel artifact for passing to `reusable_upload_wheels.yml`
+- [reusable_upload_wheels.yml](reusable_upload_wheels.yml) - This job uploads the wheels to google cloud
+  - `RRD_ARTIFACT_NAME` - Intermediate name of the GitHub rrd artifact. This should match the name passed to
+    `reusable_build_and_test_wheels.yml`
+  - `WHEEL_ARTIFACT_NAME` - Intermediate name of the GitHub wheel artifact. This should match the name passed to
+    `reusable_build_and_test_wheels.yml`
+- [reusable_build_web.yml](reusable_build_web.yml) - This job builds the wasm artifacts for the web.
+  - `RELEASE_VERSION` - If producing a release, the version number. This must match the version in `Cargo.toml`.
+- [reusable_upload_web.yml](reusable_upload_web.yml) - This job uploads the web assets to google cloud. By default this
+  only uploads to: `app.rerun.io/commit/<commit>/`
+  - `MARK_PRERELEASE_FOR_MAINLINE` - If true, then the web assets will go to `app.rerun.io/preleease/
+  - `MARK_TAGGED_VERSION` - If true, then the web assets will go to `app.rerun.io/version/<RELEASE_VERSION>`
+  - `RELEASE_VERSION` - If producing a release, the version number.
+  - `RRD_ARTIFACT_NAME` - Intermediate name of the GitHub rrd artifact. This should match the name passed to
+    `reusable_build_and_test_wheels.yml`
+  - `UPLOAD_COMMIT_OVERRIDE` - If set, will replace the value of `<commit>`. This is necessary because we want pull
+  request builds associated with their originating commit, even if the web-build happens on an ephemeral merge-commit.
+- [reusable_pr_summary.yml](reusable_pr_summary.yml) - This job updates the PR summary with the results of the CI run.
+  - This summary can be found at:
+  `https://storage.googleapis.com/rerun-builds/pull_request/<PR_NUMBER>/index.html`
+  - `PR_NUMBER` - The PR number to update. This will generally be set by the `on_pull_request.yml` workflow using:
+  `${{github.event.pull_request.number}}`
+
+## Manual Workflows
+- [manual_dispatch](manual_dispatch.yml) - This workflow is used to manually trigger the assorted reusable workflows for
+  testing.
+  - See the workflow file for the list of parameters.
+- [manual_build_wheels_for_pr.yml](manual_build_wheels_for_pr.yml) - This workflow can be dispatched on a branch and
+  will build all of the wheels for the associated pull-request. Uses:
+  - [reusable_build_and_test_wheels.yml](reusable_build_and_test_wheels.yml)
+  - [reusable_upload_wheels.yml](reusable_upload_wheels.yml)
+  - [reusable_pr_summary.yml](reusable_pr_summary.yml)

.github/workflows/manual_dispatch.yml

@@ -5,55 +5,65 @@ on:
     # NOTE: boolean inputs are still actually strings
     # See: https://github.com/actions/runner/issues/1483
     inputs:
+      # Sorted based on job workflow logic rather than alphabetical
+      CHECKS:
+        description: 'Run reuseable_checks'
+        type: boolean
+        required: false
+        default: true
+
       BENCHES:
-        description: 'Run Benches'
+        description: 'Run reusable_bench.yml'
         type: boolean
         required: false
         default: false
-      BUILD_WEB:
-        description: 'Build Web'
+
+      MIN_TEST_WHEEL:
+        description: 'Run reuseable_build_and_test_wheels (Minimal Wheel Linux Only)'
         type: boolean
         required: false
-        default: true
-      CHECKS:
-        description: 'Run All Checks'
+
+      WHEEL_PLATFORMS:
+        description: 'Run reusable_build_and_test_wheels (Full build: linux,windows,macos-arm,macos-intel)'
+        type: string
+        required: false
+        default: 'linux'
+
+      BUILD_WEB:
+        description: 'Run reusable_build_web'
         type: boolean
         required: false
         default: true
+
       DEPLOY_DOCS:
-        description: 'Deploy the docs to refs.rerun.io'
+        description: 'Run reusable_deploy_docs'
         type: boolean
         required: false
         default: false
-      MIN_TEST_WHEEL:
-        description: 'Test Minimum Wheel'
-        type: boolean
-        required: false
+
       RELEASE_VERSION:
-        description: 'Release'
+        description: 'Release Version Number (Must match Cargo.toml)'
         type: string
         required: false
         default: 'prerelease'
-      SAVE_CACHE:
-        description: 'Save Cache'
+
+      UPLOAD_GCLOUD:
+        description: 'Upload wheels and wasm to gcloud'
         type: boolean
         required: false
         default: false
-      SAVE_PR:
-        description: 'Save PR Summary'
+
+      UPDATE_PR_SUMMARY:
+        description: 'Update the PR Summary for PR#'
         type: string
         required: false
         default: ''
-      UPLOAD_GCLOUD:
-        description: 'Upload to gcloud'
+
+      SAVE_CACHE:
+        description: 'Save the rust-cache where relevant'
         type: boolean
         required: false
         default: false
-      WHEEL_PLATFORMS:
-        description: 'Platforms to build wheels for'
-        type: string
-        required: false
-        default: 'linux'
 
 jobs:
   checks:
@@ -212,8 +222,8 @@ jobs:
   save-pr-summary:
     name: 'Save PR Summary'
     needs: [upload-web, upload-wheels-linux]
-    if: ${{ github.event.inputs.SAVE_PR != '' }}
+    if: ${{ github.event.inputs.UPDATE_PR_SUMMARY != '' }}
     uses: ./.github/workflows/reusable_pr_summary.yml
     with:
-      PR_NUMBER: ${{ github.event.inputs.SAVE_PR }}
+      PR_NUMBER: ${{ github.event.inputs.UPDATE_PR_SUMMARY }}
     secrets: inherit