[GHA] Separate out hardware workflows

[kbenzie]

The goal of this patch is to make a failure of one HW build & test job only cancel jobs running on the same HW tag rather than all HW jobs for all adapters. It does this by moving the hardware jobs out of the cmake.yml workflow into the new build-hw-reusable.yml workflow, which as the name suggests is reusable. This reusable workflow is then used in by adapter specific jobs in the cmake.yml workflow to display jobs in a group which is collapsible.

[lukaszstolarczuk]

At the same time I like this change and I don't 😉 I think both of your goals are good, but I'd propose a slightly different implementation. If you feel like you still prefer your way I'm not going to be stubborn.

The reusable workflow is good, but in current approach it spawns a few more workflows per each PR. It's not that big of a deal, but instead of creating new files (per each adapter) we could do jobs per each adapter in cmake.yml, like the code below.
I think it should still achive your goals, it will omit the bloat code for each workflow file, and keep a little order in the workflows.

jobs:
  ...
  cuda:
    uses: ./.github/workflows/build-hw-reusable.yml
    with:
      name: CUDA
  hip:
    uses: ./.github/workflows/build-hw-reusable.yml
    with:
      name: HIP
  ...

Example "Checks" view on PR page:

vs

(all building workflows would be hidden as jobs in "Build and test", as it is now)

[kbenzie]

I think it should still achive your goals, it will omit the bloat code for each workflow file, and keep a little order in the workflows.

I'll try this out. I'm not 100% happy with how things got split up and displayed in my current approach either.

[kbenzie]

@lukaszstolarczuk this looks good, on here I like how the adapter jobs are also collapsable. I'm tempted to split out the other jobs in the cmake.yml in a similar way if only for ease of browsing the Build and test workflow.

[kbenzie]

One thing this approach doesn't do, which I liked about the separate workflows, is allow for retrying just the jobs for a single adapter. The whole cmake.yml workflow needs to complete before I can retry, this would be nice for when a short term network related issue pops up.