GitHub Actions to perform Kubectl Automation on my On Prem K8s Clusters

[gh-pbhandari]

Hi Team,

First of all, I really like the Actions Runner Controller (ARC) and how seamlessly the runners scale based on our requirements. I have installed it using Helm following the quick start guide https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller

Issue:

I want to automate running some Python scripts in the web container running in my on-premises Kubernetes cluster. However, I realized that the ghcr.io/actions/actions-runner:latest image that the runner uses does not have kubectl installed, which means I cannot communicate with my cluster directly from the runner.

Question:

Do we have any existing solutions for this purpose, or is creating a custom Docker image with kubectl installed the recommended approach? If custom images are the way to go, could anyone share best practices or examples for creating and using such custom runner images?

Any information or guidance would be greatly appreciated.

Thanks in advance!

[nikola-jokic]

Hi @gh-pbhandari,

Thank you for your kind words! You are right, creating a custom docker image would be the right approach.
Here is the Dockerfile we use to build an image, but you can start building your own by having ours as a base image. It depends on your preference ☺️

[gh-pbhandari]

@nikola-jokic : Thanks for getting back.
Can you point me to the location where this custom image would go as part of the helm chart ?

Just this is enough ? https://github.com/actions/actions-runner-controller/blob/master/charts/gha-runner-scale-set/values.yaml#L124

[nikola-jokic]

Yes, the image field of the container that is named runner is the place where you specify your custom image ☺️

[gh-pbhandari]

@nikola-jokic : Looks like we don't have ImagePull Secrets implemented to pull image from Private repositories.. I will go ahead and update the chart locally.

Is this true or do we have some way to pull from Private Repository for Custom Images ?

Sample DockerFile

# Use the base image for the GitHub Actions runner
FROM ghcr.io/actions/actions-runner:latest

# Install sudo, curl, and kubectl
USER root
RUN apt-get update && apt-get install -y sudo curl \
    && curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" \
    && chmod +x kubectl \
    && mv kubectl /usr/local/bin/ \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

# Switch back to the default user
USER runner

[gh-pbhandari]

nvm.. It said, the imagePullPolicy Inherits from the Controller and I was able to get it up and running. Now, just have to test the custom image..

ButI just wanted to report that when we install the Controller, the deployment name is arc-gha-rs-controller but when we deploy the runner, it expects the name to be gha-rs-controller instead.

helm template arc-runner-set gha-runner-scale-set -n arc-runner-set
Error: execution error at (gha-runner-scale-set/templates/manager_role_binding.yaml:17:11): No gha-rs-controller deployment found using label (app.kubernetes.io/part-of=gha-rs-controller). Consider setting controllerServiceAccount.name in values.yaml to be explicit if you think the discovery is wrong.

Use --debug flag to render out invalid YAML

To Install, I had to explicitly define the controllerServiceAccount.name

[gh-pbhandari]

I was able to use Custom Image and deploy the Arc Runner Set successfully. But I would like to ask the Behavior of minimum runners. I did keep a minimum of 1 runner for my testing so that the overall execution time of actions is reduced.

But I noticed that even though I have minimum runner, it deployed a new one to get the job done and deleted the already running Runner.

NAME                                READY   STATUS    RESTARTS   AGE
arc-runner-set-rkk8w-runner-vfxv6   1/1     Running   0          17s
arc-runner-set-rkk8w-runner-dw58v   0/1     Pending   0          0s
arc-runner-set-rkk8w-runner-dw58v   0/1     ContainerCreating   0          0s
arc-runner-set-rkk8w-runner-dw58v   1/1     Running             0          2s
arc-runner-set-rkk8w-runner-vfxv6   0/1     Completed           0          5m17s
arc-runner-set-rkk8w-runner-vfxv6   0/1     Terminating         0          5m19s

What's the point of minimum runner ?