Running Prisma Cloud Defender on Bottlerocket

[netjordan]

Hello,

We use a container vulnerability scanning tool called Prisma Cloud (formally known as twistlock). I've tried running this as a DaemonSet on bottle rocket, but are getting some errors. I wondered if anyone had tried to run any similar tools or can see something obviously wrong with this configuration?

I have tried the following paths for DOCKER_CLIENT_ADDRESS:

• /var/run/docker.sock
• /run/dockershim.sock
• /run/containerd.sock
• /run/containerd/containerd.sock
• /run/containerd.sock

In the example below, using /run/dockershim.sock - I get the following errors:
WARN 2021-08-06T08:54:31.834 defender.go:1954 Failed to get docker client with address /run/dockershim.sock. err Get "http://unix.sock/version": net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"

With other configurations, I get file not found errors.

Example Daemonset:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: twistlock-defender-ds
  namespace: twistlock
spec:
  selector:
    matchLabels:
      app: twistlock-defender
  template:
    metadata:
      labels:
        app: twistlock-defender
      annotations:
        container.apparmor.security.beta.kubernetes.io/twistlock-defender: unconfined
    spec:
      volumes:
        - name: certificates
          secret:
            secretName: twistlock-secrets
            defaultMode: 256
            optional: false
        - name: syslog-socket
          hostPath:
            path: /dev/log
            type: ''
        - name: data-folder
          hostPath:
            path: /var/lib/twistlock
            type: ''
        - name: docker-netns
          hostPath:
            path: /var/run/docker/netns
            type: ''
        - name: passwd
          hostPath:
            path: /etc/passwd
            type: ''
        - name: docker-sock-folder
          hostPath:
            path: /run/
            type: ''
        - name: auditd-log
          hostPath:
            path: /var/log/audit
            type: ''
        - name: iptables-lock
          hostPath:
            path: /run
            type: ''
      containers:
        - name: twistlock-defender
          image: >-
            registry-auth.twistlock.com/blah/twistlock/defender:defender_21_04_421
          env:
            - name: WS_ADDRESS
              value: 'wss://twistlock:443'
            - name: DEFENDER_TYPE
              value: daemonset
            - name: DEFENDER_LISTENER_TYPE
              value: none
            - name: LOG_PROD
              value: 'true'
            - name: SYSTEMD_ENABLED
              value: 'false'
            - name: DOCKER_CLIENT_ADDRESS
              value: /run/dockershim.sock
            - name: DEFENDER_CLUSTER_ID
              value: 12345
            - name: DEFENDER_CLUSTER
              value: infrastructure-dev
            - name: MONITOR_SERVICE_ACCOUNTS
              value: 'true'
            - name: MONITOR_ISTIO
              value: 'false'
            - name: COLLECT_POD_LABELS
              value: 'false'
            - name: INSTALL_BUNDLE
              value: >-
                blah
            - name: HOST_CUSTOM_COMPLIANCE_ENABLED
              value: 'true'
          resources:
            limits:
              memory: 512Mi
            requests:
              cpu: 256m
          volumeMounts:
            - name: data-folder
              mountPath: /var/lib/twistlock
              mountPropagation: None
            - name: certificates
              mountPath: /var/lib/twistlock/certificates
              mountPropagation: None
            - name: docker-sock-folder
              mountPath: /run/
              mountPropagation: None
            - name: passwd
              readOnly: true
              mountPath: /etc/passwd
              mountPropagation: None
            - name: docker-netns
              readOnly: true
              mountPath: /var/run/docker/netns
              mountPropagation: None
            - name: syslog-socket
              mountPath: /dev/log
              mountPropagation: None
            - name: auditd-log
              mountPath: /var/log/audit
              mountPropagation: None
            - name: iptables-lock
              mountPath: /run
              mountPropagation: None
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              add:
                - NET_ADMIN
                - NET_RAW
                - SYS_ADMIN
                - SYS_PTRACE
                - SYS_CHROOT
                - MKNOD
                - SETFCAP
            privileged: false
            runAsNonRoot: false
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccountName: twistlock-service
      serviceAccount: twistlock-service
      automountServiceAccountToken: true
      hostNetwork: true
      hostPID: true
      shareProcessNamespace: false
      securityContext: {}
      schedulerName: default-scheduler
      enableServiceLinks: true
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  revisionHistoryLimit: 10

Thanks,
Jordan

[netjordan]

I think I have solved this myself, for anyone wondering the following options can be chosen when generating the deployment manifest from the prisma console.

[jhaynes]

Glad you found a way to make Prisma Cloud work!

I suspect the critical bit is the Nodes use Container Runtime Interface (CRI), not Docker in the screenshot you provided. Bottlerocket's Kubernetes variants all use containerd rather than Docker. The containerd socket (for Kubernetes variants) is at /run/dockershim.sock which I think you tried so I was a little surprised to see it didn't work. We moved it there so applications (like this one) have a higher chance of working out-of-the-box on Bottlerocket.

Poking around at Prisma's docs and following a link, I think that checking that cri box will change the yaml file to be:

env:
- name: DEFENDER_TYPE
   value: cri

vs value: daemonset.

[hi-artem]

@netjordan Is your Host monitoring still works in bottlerocket? We have observed a major degradation in Prisma Cloud Compute feature set after migrating. In particular, all of the Host Activity enforcement is no longer working.