Hashicorp vault root token causing keda operator panic

[kmdrn7]

Report

Panic error observed from keda operator while using ScaledObject with TriggerAuthentication that use hashicorp vault provider authenticated using root token.

While using non-root token for authenticating to vault, keda operator running just fine

Expected Behavior

Expecting keda operator running without error while using TriggerAuthentication that use hashicorp vault provider authenticated using root token

Actual Behavior

Keda operator throwing panic error and entering CrashLoopBackOff state

Steps to Reproduce the Problem

1. deploy Deployment resource
2. deploy TriggerAuthentication with hashicorp vault provider

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: webserver
spec:
  hashiCorpVault:
    address: http://vault.default:8200
    authentication: token
    credential:
      token: ====the-token====
    secrets:
    - key: value
      parameter: connection
      path: secret/webserver/pgconn

3. deploy ScaledObject

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: webserver
spec:
  scaleTargetRef:
    name: webserver
  triggers:
    - type: postgresql
      authenticationRef:
        kind: TriggerAuthentication
        name: webserver
      metadata:
        targetQueryValue: "1"
        query: >-
          SELECT ceil(COUNT(*)::decimal / 16)
          FROM task_instance
          WHERE (state='running' OR state='queued')

Logs from KEDA operator

2023-11-19T11:49:45Z    INFO    Reconciling ScaledObject    {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"webserver","namespace":"default"}, "namespace": "default", "name": "webserver", "reconcileID": "4ebacfb0-c9a8-4ad8-bd95-d98ee1e80f70"}
2023-11-19T11:49:45Z    INFO    Creating a new HPA    {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"webserver","namespace":"default"}, "namespace": "default", "name": "webserver", "reconcileID": "4ebacfb0-c9a8-4ad8-bd95-d98ee1e80f70", "HPA.Namespace": "default", "HPA.Name": "keda-hpa-webserver"}
2023-11-19T11:49:45Z    INFO    cert-rotation    Ensuring CA cert    {"name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService", "name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService"}
2023-11-19T11:49:45Z    INFO    Observed a panic in reconciler: interface conversion: interface {} is nil, not bool    {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"webserver","namespace":"default"}, "namespace": "default", "name": "webserver", "reconcileID": "4ebacfb0-c9a8-4ad8-bd95-d98ee1e80f70"}
panic: interface conversion: interface {} is nil, not bool [recovered]
    panic: interface conversion: interface {} is nil, not bool
goroutine 282 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:115 +0x1fa
panic({0x3874f60, 0xc00159ccf0})
    /usr/local/go/src/runtime/panic.go:884 +0x213
github.com/kedacore/keda/v2/pkg/scaling/resolver.(*HashicorpVaultHandler).Initialize(0xc00114a120, {{0x4754cc8?, 0xc0013cf2f0?}, 0xc0009fc5a0?})
    /workspace/pkg/scaling/resolver/hashicorpvault_handler.go:76 +0x26d
github.com/kedacore/keda/v2/pkg/scaling/resolver.resolveAuthRef({0x474d110, 0xc000971440}, {0x4764880, 0xc0009fc5a0}, {{0x4754cc8?, 0xc0013cf2f0?}, 0x40dcca?}, 0xc0004f46a0, 0xc0011e60e8, {0xc00042f5c0, ...}, ...)
    /workspace/pkg/scaling/resolver/scale_resolvers.go:243 +0x337
github.com/kedacore/keda/v2/pkg/scaling/resolver.ResolveAuthRefAndPodIdentity({0x474d110, 0xc000971440}, {0x4764880?, 0xc0009fc5a0?}, {{0x4754cc8?, 0xc0013cf2f0?}, 0x4754cc8?}, 0xc0013cf2f0?, 0xc0011e6000, {0xc00042f5c0, ...}, ...)
    /workspace/pkg/scaling/resolver/scale_resolvers.go:182 +0xc9
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).buildScalers.func1()
    /workspace/pkg/scaling/scalers_builder.go:68 +0x3f4
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).buildScalers(0xc000d43b20, {0x474d110?, 0xc000971440}, 0xc00058b680, 0xc0011e6000, {0x0, 0x0})
    /workspace/pkg/scaling/scalers_builder.go:78 +0x5d6
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).performGetScalersCache(0xc000d43b20, {0x474d110, 0xc000971440}, {0xc0010fb3e0, 0x1e}, {0x3e7de80, 0xc000f9c000}, 0xc000f9a8f8, {0x0, 0x0}, ...)
    /workspace/pkg/scaling/scale_handler.go:360 +0x71c
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).GetScalersCache(0xc0005a6400?, {0x474d110, 0xc000971440}, {0x3e7de80, 0xc000f9c000})
    /workspace/pkg/scaling/scale_handler.go:281 +0xf6
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).getScaledObjectMetricSpecs(0xc0006006c0, {0x474d110, 0xc000971440}, {{0x4754cc8?, 0xc0009714a0?}, 0xc14e9b76691f7931?}, 0xc0005a6400)
    /workspace/controllers/keda/hpa.go:209 +0xda
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).newHPAForScaledObject(0xc0006006c0, {0x474d110?, 0xc000971440?}, {{0x4754cc8?, 0xc0009714a0?}, 0xc00159bd80?}, 0xc0005a6400, 0xc000d5b0e0)
    /workspace/controllers/keda/hpa.go:75 +0x66
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).createAndDeployNewHPA(0xc0006006c0, {0x474d110, 0xc000971440}, {{0x4754cc8?, 0xc0009714a0?}, 0xc000971440?}, 0xc0005a6400, 0x0?)
    /workspace/controllers/keda/hpa.go:48 +0x1d6
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).ensureHPAForScaledObjectExists(0xc0006006c0, {0x474d110, 0xc000971440}, {{0x4754cc8?, 0xc0009714a0?}, 0x4754cc8?}, 0xc0005a6400, 0xc00009a400?)
    /workspace/controllers/keda/scaledobject_controller.go:394 +0x356
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).reconcileScaledObject(0xc0006006c0?, {0x474d110, 0xc000971440}, {{0x4754cc8?, 0xc0009714a0?}, 0xc00042f5a6?}, 0xc0005a6400, 0xc0004a8c00)
    /workspace/controllers/keda/scaledobject_controller.go:254 +0x845
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).Reconcile(0xc0006006c0, {0x474d110, 0xc000971440}, {{{0xc00042f5c0?, 0x0?}, {0xc00042f5a6?, 0x40e0a7?}}})
    /workspace/controllers/keda/scaledobject_controller.go:177 +0x555
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x474d110?, {0x474d110?, 0xc000971440?}, {{{0xc00042f5c0?, 0x3610820?}, {0xc00042f5a6?, 0x472f3f0?}}})
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:118 +0xc8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0004ddb80, {0x474d068, 0xc000ceaff0}, {0x3a332e0?, 0xc0004f40c0?})
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:314 +0x377
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0004ddb80, {0x474d068, 0xc000ceaff0})
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:265 +0x1d9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:226 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222 +0x587

KEDA Version

2.11.2

Kubernetes Version

1.26

Platform

Google Cloud

Scaler Details

Postgresql

Anything else?

No response

[zroubalik]

Hi @kmdrn7 , thanks for reporting, is this a duplicate of #4964 ?

[kmdrn7]

I think it's different error, this error is related to nil interface{} to boolean casting, while #4964 is related to missing credential value