[wiz-kubernetes-connector] Connector secret created by Helm chart is not accepted by deployment

[alfredkrohmer]

See here:

• https://github.com/wiz-sec/charts/blob/master/wiz-kubernetes-connector/templates/secret-connector.yaml#L17
• https://github.com/wiz-sec/charts/blob/master/wiz-kubernetes-connector/templates/_helpers.tpl#L50-L58

This creates the connector secret with several keys, however the deployment requires a single connectorData key (supposedly in JSON format) instead: https://github.com/wiz-sec/charts/blob/master/wiz-kubernetes-connector/templates/wiz-broker-deployment.yaml#L38-L43

Also, when adding a Kubernetes connector in the wiz.io console, a kubectl command is shown to create this secret manually, this also uses multiple keys in the secret instead of a single connectorData key.

[uri-peled-unit]

We are experiencing similar problems with the connector secret:

Unable to attach or mount volumes: unmounted volumes=[connector-data], unattached volumes=[connector-data kube-api-access-hv47p]: timed out waiting for the condition

MountVolume.SetUp failed for volume "connector-data" : references non-existent secret key: connectorData

[lir-wiz]

Hey @alfredkrohmer and @uri-peled-unit
Can you help me validate my thought that this only happens when setting autoCreateConnector: false?
Is this the case for your installation? If so, why did you set it?
Thanks!

[uri-peled-unit]

Hey @lir-wiz ,
Thanks for the reply I just had a call with Shifer about the same issue:

About your question. Actually, I set it to true, and it is still not working for us,
please see our values below:

        global:
          wizApiToken:
            secret:
              create: false
              name: "wiz-api-token"
        wiz-kubernetes-connector:
          enabled: true
          broker:
            enabled: true
          wizConnector:
            autoCreateConnector: 
              enabled: true
              clusterFlavor: EKS
            createSecret: true
            secretName: wiz-connector-autocreated-secret
        wiz-sensor:
          enabled: true
          imagePullSecret:
            create: false
            name: "wiz-sensor-imagepullkey"
        wiz-admission-controller:
          enabled: true

[lir-wiz]

autoCreateConnector should be one level higher, like so:

global:
  wizApiToken:
    secret:
      create: false
      name: "wiz-api-token"
wiz-kubernetes-connector:
  enabled: true
  broker:
    enabled: true
  autoCreateConnector: 
    enabled: true
    clusterFlavor: EKS
  wizConnector:
    createSecret: true
    secretName: wiz-connector-autocreated-secret
wiz-sensor:
  enabled: true
  imagePullSecret:
    create: false
    name: "wiz-sensor-imagepullkey"
wiz-admission-controller:
  enabled: true

Can you try again using those values and let me know if it works please?
Make sure to uninstall the chart beforehand, to make sure we do a clean test.
Thanks

[uri-peled-unit]

Yes, I tried this as well with the above fix. and the same issue is happening

[dkosev-yotpo]

@lir-wiz I'm observing an identical issue with my setup. The expected key connectorData is actually defined in a secret with an identical name in wiz-broker. That might be causing the confusion. Here's my values file for clarity:

wizApiToken:
  clientId: "REDACTED"
  clientToken: "REDACTED"
  clientEndpoint: ""

broker:
  enabled: true

autoCreateConnector:
  connectorName: "REDACTED"
  clusterFlavor: "EKS"

Resulting error:

[lir-wiz]

This issue can happen if the connector creation Job fails. Can you please verify that the Job ran successfully?

[dkosev-yotpo]

After troubleshooting and fixing the connector creation Job, the connector secret was created successfully and subsequently mounted. Thanks!

[kvenu]

I am facing a similar issue while deploying wiz helm chart on k8s clusters. I get the below error when enabling autoCreateConnector. I cannot see any logs and not sure where the problem is. I am using 2.4.4 helm chart with wiz-broker:2.4 image. The same chart works in few other kubernetes clusters, but failing on few

State:          Terminated
  Reason:       Error
  Exit Code:    132

$ kubectl get pod
NAME READY STATUS RESTARTS AGE
wiz-kubernetes-connector-create-connector-6pgpw 0/1 Error 0 46m
wiz-kubernetes-connector-create-connector-8kl59 0/1 Error 0 46m

[sherifkayad]

@lir-wiz I have the same exact issue like explained by @kvenu .. can you please elaborate how this can happen / what a potential cause might be? .. The weird thing is that it's okay in a couple of clusters and only one cluster is problematic ..

[miltlima]

@lir-wiz how are you doing ?

I have been following the wiz documentation to deploy in our clusters but this showing this error

{"level":"info","time":"2024-05-17T15:03:22.402277833Z","msg":"Creating kubernetes connector","apiServerEndpoint":"https://kubernetes.default.svc.cluster.local","serviceType":"GKE","connectorName":"","clusterExternalID":"f41b840d06cc04ad4b6c0cdcd379c79afe2936d9e96e4320892d1a3068274daf","isOnPrem":true}
Error: failed creating kubernetes connector: rpc error: code = Unauthenticated desc = Unauthorized - Service account accessed unauthorized service
{"level":"fatal","time":"2024-05-17T15:03:22.506388673Z","msg":"Failed executing entrypoint","error":"failed creating kubernetes connector: rpc error: code = Unauthenticated desc = Unauthorized - Service account accessed unauthorized service"}

I am using this values:

global:
  wizApiToken:
    secret:
      create: false
      name: "wiz-api-token"
wiz-kubernetes-connector:
  enabled: true
  autoCreateConnector:
    enabled: true
    clusterFlavor: GKE
  broker:
    enabled: true
    serviceAccount:
      create: true
      name: "wiz-broker"
  wizConnector:
    createSecret: true
    secretName: wiz-connector-autocreated-secret
wiz-sensor:
  enabled: true
  imagePullSecret:
    create: false
    name: "wiz-sensor-imagepullkey"
wiz-admission-controller:
  enabled: true
  webhook:
    errorEnforcementMethod: "AUDIT"
    policyEnforcementMethod: "AUDIT"

[ofirc-wiz]

Hi @miltlima, Ofir Cohen from the Kubernetes team in Wiz.
It looks like either of the following:

1. The Wiz Service Account that you are using doesn't exist
2. The Wiz Service Account credentials that have provided are invalid
3. The Wiz Service Account type that you are using is incorrect

If you want to be on the safe side and keep things stupid and simple, I would suggest to do the following:
0. Backup your existing cluster settings: kubectl get secret wiz-api-token -nwiz > wiz-api-token.yaml

1. Wiz Portal > Settings > Access Management > Service Accounts > Add Service Account > Type: Complete Kubernetes Integration
2. Update the token in your secret, kubectl edit secret wiz-api-token -n wiz
3. Trigger a rolling update of your deployments kubectl rollout restart deployment deployment-name -n wiz

If that still doesn't work, let's continue on the official channels and assist you with this issue.

[miltlima]

Thanks @ofirc-wiz I solved it, the SA role was wrong !

[lamadome]

I am facing the same issue. After changing the helper file I made it work. Also added elements to the chart

{{- define "wiz-kubernetes-connector.wizConnectorSecretData" -}}
{{- if not .Values.autoCreateConnector.enabled }}
connectorData: |
  {
    "ConnectorId": {{ required "A valid .Values.wizConnector.connectorId entry required!" .Values.wizConnector.connectorId | quote }},
    "TunnelToken": {{ required "A valid .Values.wizConnector.connectorToken entry required!" .Values.wizConnector.connectorToken | quote }},
    "TunnelDomain": {{ required "A valid .Values.wizConnector.targetDomain entry required!" .Values.wizConnector.targetDomain | quote }},
    "TunnelServerDomain": {{ required "A valid .Values.wizConnector.tunnelserverDomain entry required!" .Values.wizConnector.tunnelserverDomain | quote }},
    "TunnelServerPort": {{ required "A valid .Values.wizConnector.tunnelserverPort entry required!" .Values.wizConnector.tunnelserverPort | quote }}
  }
{{- end }}
{{- end }}

the values file now looks like this:

wizConnector:
   createSecret: true
   targetIp: "10.85.241.92"
   targetPort: "6443"
   connectorId: "d44171df-186d-4a3d-aa76-68477987c001"
   connectorToken: "a4b5e3e5ca62f99815b472a1561fdf09ac97ea82fd75f3c78e439d3b6c73c29b"
   targetDomain: "3fca60e8-59e4-4e80-8ea6-53df96c04abb.tunnel.wiz.io"
   tunnelserverDomain: "tunnel.eu3.test.wiz.io" 
   tunnelserverPort: "443"

TargetIP can be set under broker.
Then by using

global
  podCustomEnvironmentVariables:
      - name: TARGET_PORT
         value: "6443"

I was able to set the port separately