How are you managing multiple Runner Scale Sets?

[jb-2020]

Hey folks, curious how users of Runner Scale Sets (RSS) are managing these objects. We use Flux v2 for GitOps and therefore define a HelmRelease for each RSS.

We're running into some administration headaches around the values passed into this chart:

• We use RSS in org mode, therefore we have multiple RSS(s) per org (size, architecture, etc).
• We override the default PodSpec with an init container, resource requests, container images, etc. This means the helm value overrides are over 100 lines long.

As you can probably guess, this is ripe for configuration skew. We've tried Kustomize and are now looking into getting clever with helm.

How are you managing multiple RSS? Any suggestions? I think if ARC had the capability to 'merge' overrides to the PodSpec then some of these painpoints would be avoided. With our current approach, we need to also keep a close eye on any changes to the pod spec (e.g. Docker socket location).

[mrclrchtr]

Hi, I'm using kustomize:

Base config in base/values.yaml and specific config with patches:

overlays/resources-high/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

helmCharts:
  - name: gha-runner-scale-set
    repo: oci://ghcr.io/actions/actions-runner-controller-charts
    releaseName: runner
    namespace: gha-rss-resources-high
    version: 0.9.2
    valuesFile: ../../base/values.yaml
    # noinspection KubernetesUnknownKeys
    additionalValuesFiles:
      - values.yaml

resources:
  - namespace.yaml
  - ../../base

patches:
  - path: service-account.yaml
  - target:
      group: actions.github.com
      version: v1alpha1
      kind: AutoscalingRunnerSet
      name: k8s-runner-high
    path: values-patch.yaml

overlays/resources-high/values.yaml

runnerScaleSetName: k8s-runner-high

overlays/resources-high/values-patch.yaml

- op: add
  path: /spec/minRunners
  value: 3
- op: add
  path: /spec/maxRunners
  value: 3
- op: add
  path: /spec/template/spec/containers/0/resources
  value:
    limits:
      cpu: 3000m
      memory: 5Gi
- op: add
  path: /spec/template/spec/topologySpreadConstraints/0/labelSelector/matchLabels/actions.github.com~1scale-set-name
  # noinspection YAMLIncompatibleTypes
  value:
    k8s-runner-high
- op: add
  path: /spec/template/spec/volumes/0/ephemeral/volumeClaimTemplate/spec/resources
  value:
    requests:
      storage: 3Gi