Bug: Failed to check if runner is busy

[callum-tait-pbx]

This has been mentioned in a few places, I think it would be good to have an issue raised specifically for this going forward.

Steps to Reproduce

1. Deploy your chosen auth method
2. cd config
3. kustomize build ./default > .manifest.yaml
4. kubectl apply -f ./manifest.yaml
5. Deploy your runner configuration, an example is documented below:
6. Check manager logs kubectl logs controller-manager-777fb76ddb-4tjpz -c manager

Error

2020-11-12T23:24:07.206Z  ERROR  controllers.Runner. Failed to check if runner is busy. {"runner": "actions-runner-system/runner-deployment", "error": "runner not found"}

Behaviour
This doesn't appear to prevent the runner being created nor scaling from working however there is a problem somewhere as we shouldn't be getting errors thrown at all

Example Runner Yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: runners-service-account
  namespace: actions-runner-system
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: runner-deployment
  namespace: actions-runner-system
spec:
  replicas: 1
  template:
    spec:
      organization: YOUR_ORG
      serviceAccountName: runners-service-account
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: runner-deployment-autoscaler
  namespace: actions-runner-system
spec:
  scaleDownDelaySecondsAfterScaleUp: 10
  scaleTargetRef:
    name: runner-deployment
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
    repositoryNames:
    - some-repo
    - some-repo-2

[mumoshu]

@callum-tait-pbx I think this has been disappeared in my env once I've added actions:read permission to my github app. Could you confirm?

[callum-tait-pbx]

Currently deployed with a PAT which doesn't have a actions:read scope as I was having issues with my app for some reason. I'll redeploy with the app setup and confirm.

[callum-tait-pbx]

Just deployed again using 0.11.2 with the github app auth configuration at the organisation level. I still get the error (see below. I can confirm permission wise my app has:

Repo Permissions

• Actions | Read only
• Administration | Read & Write
• Metadata | Read

Org Permissions

• Actions | Read & Write

2020-11-13T13:01:03.904Z	ERROR	controllers.Runner	Failed to check if runner is busy	{"runner": "sre-actions-runner-system/sre-actions-runners-runner-deployment-k2j7j-ctpbz", "error": "runner not found"}

Again my runner seems to actually have deployed but it would be good to figure out why this error happens and fix whatever is causing it.

[jaeseok-moloco]

same issue for me.

log:
2020-11-27T15:17:14.629Z ERROR controller-runtime.controller Reconciler error {"controller": "runner", "request": "default/self-hosted-1x-deployment-9gb6x-gj2l4", "error": "runner is busy"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190913080033-27d36303b655/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190913080033-27d36303b655/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190913080033-27d36303b655/pkg/util/wait/wait.go:88

2020-11-27T15:17:21.506Z ERROR controllers.Runner Failed to unregister runner {"runner": "default/self-hosted-1x-deployment-dfgwp-hbc7d", "error": "runner is busy"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
github.com/summerwind/actions-runner-controller/controllers.(*RunnerReconciler).Reconcile
/workspace/controllers/runner_controller.go:98
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190913080033-27d36303b655/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190913080033-27d36303b655/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190913080033-27d36303b655/pkg/util/wait/wait.go:88

I'm using the runner for my organization.

My app permissions:

• Actions: R&W

• Administration: R&W

• Metadata: R

• Workflows: R&W

• Self-hosted runners: R&W

Actions-runner-controller

• version: v0.13.1

Container Setting

• Image: summerwind/actions-runner-dind:latest
• ImagePullPolicy: Always
• Image Id: docker-pullable://summerwind/actions-runner-dind@sha256:76a1fcb9803a814e812673958b0ae4ab70dc6bf600ab5b7ff72f48dc9ca340aa

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[callum-tait-pbx]

I no longer get this on the controller v0.18.0, I'm not sure when we stopped getting the error, this was probably fixed at some point by accident. If this is an issue for someone on this thread still, upgrade your controller to latest. If you still get the error raise a new issue and link to this one to provide addiitonal context.