CKS Exam Preparation

• CKS Exam Preparation
  • Intro
  • Usefull courses
  • General security-related docs
  • Cirriclium Topics
  • Cluster Setup – 10%
    • Use Network security policies to restrict cluster level access
    • Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)
    • Properly set up Ingress objects with security control
    • Protect node metadata and endpoints
    • Minimize use of, and access to, GUI elements
    • Verify platform binaries before deploying
  • Cluster Hardening – 15%
    • Restrict access to Kubernetes API
    • Use Role Based Access Controls to minimize exposure
    • Exercise caution in using service accounts e.g. disable defaults, minimize permissions on newly created ones
    • Update Kubernetes frequently
  • System Hardening – 15%
    • Minimize host OS footprint (reduce attack surface)
    • Minimize IAM roles
    • Minimize external access to the network
    • Appropriately use kernel hardening tools such as AppArmor, seccomp
  • Minimize Microservice Vulnerabilities – 20%
    • Setup appropriate OS level security domains e.g. using PSP, OPA, security contexts
    • Manage Kubernetes secrets
    • Use container runtime sandboxes in multi-tenant environments (e.g. gvisor, kata containers)
    • Implement pod to pod encryption by use of mTLS
  • Supply Chain Security – 20%
    • Minimize base image footprint
    • Secure your supply chain: whitelist allowed registries, sign and validate images
    • Use static analysis of user workloads (e.g.Kubernetes resources, Docker files)
    • Scan images for known vulnerabilities
  • Monitoring, Logging and Runtime Security – 20%
    • Perform behavioral analytics of syscall process and file activities at the host and container level to detect malicious activities
    • Detect threats within physical infrastructure, apps, networks, data, users and workloads
    • Detect all phases of attack regardless where it occurs and how it spreads
    • Perform deep analytical investigation and identification of bad actors within environment
    • Ensure immutability of containers at runtime
    • Use Audit Logs to monitor access
  • Uncategorized and questions
  • Related links

Intro

In order to take the CKS exam, you must have Valid CKA certification to demonstrate you possess sufficient Kubernetes expertise. If you do not have passed CKA exam, here you find my learn path for that: CKALearn As firt to understand are you for that exam or not plese try to do that tas: Securing a Cluster of the official K8s documentation.

Usefull courses

• Linux Academy: Kubernetes Security (Advanced Concepts)
• Linux Academy: Kubernetes Security

General security-related docs

• [K8s Blog] 11 Ways (Not) to Get Hacked https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/
• GCP (GKE) General security guide https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster
• GCP (GKE) General security overview https://cloud.google.com/kubernetes-engine/docs/concepts/security-overview

Cirriclium Topics

Cluster Setup – 10%

Use Network security policies to restrict cluster level access

• Main doc: https://kubernetes.io/docs/concepts/services-networking/network-policies/
• Main task: https://kubernetes.io/docs/tasks/administer-cluster/declare-network-policy/
• General practice: https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-network-access
• Official blog post: https://kubernetes.io/blog/2017/10/enforcing-network-policies-in-kubernetes/
• NetworkPolicy API object reference: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#networkpolicy-v1-networking-k8s-io

3rd Party:

• NP Examples: https://github.com/ahmetb/kubernetes-network-policy-recipes
• Anthos security blueprint: Restricting traffic - example approaches and implementation steps https://github.com/GoogleCloudPlatform/anthos-security-blueprints/tree/master/restricting-traffic
• Good NP description with examples: https://medium.com/@reuvenharrison/an-introduction-to-kubernetes-network-policies-for-security-people-ba92dd4c809d
• NP best practices: https://medium.com/@tufin/best-practices-for-kubernetes-network-policies-2b643c4b1aa
• [Playground] Network Policy Visualizer https://orca.tufin.io/netpol/

• Notes:

  podSelector: This selects particular Pods in the same namespace as the NetworkPolicy which should be allowed as ingress sources or egress destinations.

Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)

3rd Party:

• CIS Benchmark Kubernetes https://www.cisecurity.org/benchmark/kubernetes/
• kubebench (CNCF) https://github.com/aquasecurity/kube-bench#running-kube-bench
• Default GKE cluster results: https://cloud.google.com/kubernetes-engine/docs/concepts/cis-benchmarks#status

Properly set up Ingress objects with security control

• Main doc (TLS): https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/tls.md
• Ingress SSL Ciphers and other SSL settings https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/configmap.md#ssl-ciphers
• Main Concept (ingress, TLS): https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
• How to deploy NGINX Ingress Controller: https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md
• Main Concept (ingress controller, multiple controllers): https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
• Create TLS secret: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-secret-tls-em-

Protect node metadata and endpoints

• General: https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access
• Kubelet authentication/authorization (access node info via kubelet API) https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-authentication-authorization/
• Set Kubelet parameters via a config file https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/

 

3rd Party:

• Kubelet API https://www.deepnetwork.com/blog/kubernetes/2020/01/13/kubelet-api.html
• [Practical] Protecting metadata - iptables rule: https://docs.aws.amazon.com/eks/latest/userguide/restrict-ec2-credential-access.html
• GCP-specific metadata protection guide https://cloud.google.com/kubernetes-engine/docs/how-to/protecting-cluster-metadata
• Setting up secure endpoints in Kubernetes (might be not related): https://blog.cloud66.com/setting-up-secure-endpoints-in-kubernetes/
• Falco webinar (just a demo): Intro to Falco: Intrusion Detection for Containers - Shane Lawrence, Shopify

• Notes:

  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 169.254.169.254/32

Minimize use of, and access to, GUI elements

• Main doc: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
• Dashboard Access control: https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/README.md
• Dashboard auth Step-by-Step: https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md

 

3rd Party:

• [Long Read] On Securing the Kubernetes Dashboard: https://blog.heptio.com/on-securing-the-kubernetes-dashboard-16b09b1b7aca

Verify platform binaries before deploying

• K8s Releases (with SHA checksums): https://github.com/kubernetes/kubernetes/releases

 

3rd Party:

• sha256sum (https://help.ubuntu.com/community/HowToSHA256SUM)

Cluster Hardening – 15%

Main doc (and beyond): https://kubernetes.io/docs/reference/access-authn-authz/

 

3rd Party:

• Kubernetes RBAC and TLS certificates – Kubernetes security guide (part 1). https://sysdig.com/blog/kubernetes-security-rbac-tls/

Restrict access to Kubernetes API

• Main doc: https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/
• Cleate "Normal User" (including certificate and its signing by API) https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#normal-user
• Different ways for creation certificates (easyrsa, openssl, cfssl ) https://kubernetes.io/docs/concepts/cluster-administration/certificates/

 

3rd Party:

• Restrict access to API via NP: https://medium.com/@tufin/protecting-your-kubernetes-api-server-5eefeea4cf8a

Use Role Based Access Controls to minimize exposure

• Main doc: https://kubernetes.io/docs/reference/access-authn-authz/rbac/

 

3rd Party:

• [Practice] RBAC, PSP, NP, TLS, etc. https://github.com/David-VTUK/CKA-StudyGuide/blob/master/RevisionTopics/Part-5-Security.md

Exercise caution in using service accounts e.g. disable defaults, minimize permissions on newly created ones

• Main doc: https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/
• [Task] Service Account use (+automountServiceAccountToken): https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server
• Default Roles: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings
• Auth Modules: https://kubernetes.io/docs/reference/access-authn-authz/authorization/#authorization-modules

 

3rd Party:

• [Youtube] Understand Role Based Access Control in Kubernetes https://www.youtube.com/watch?v=G3R24JSlGjY
• Get SA token: https://docs.armory.io/docs/armory-admin/manual-service-account/
• Blogpost series:
  • [1/4] A Primer on Kubernetes Access Control https://thenewstack.io/a-primer-on-kubernetes-access-control/
  • [2/4] A Practical Approach to Understanding Kubernetes Authentication https://thenewstack.io/a-practical-approach-to-understanding-kubernetes-authentication/
  • [3/4] A Practical Approach to Understanding Kubernetes Authorization https://thenewstack.io/a-practical-approach-to-understanding-kubernetes-authorization/
  • [4/4] Kubernetes Access Control: Exploring Service Accounts https://thenewstack.io/kubernetes-access-control-exploring-service-accounts/
• Securing Kubernetes Clusters by Eliminating Risky Permissions: https://www.cyberark.com/resources/threat-research-blog/securing-kubernetes-clusters-by-eliminating-risky-permissions

Update Kubernetes frequently

• Main doc (kubeadm upgrade): https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
• Reference (kubeadm upgrade): https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-upgrade/

System Hardening – 15%

Minimize host OS footprint (reduce attack surface)

• [K8s] Preventing containers from loading unwanted kernel modules https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#preventing-containers-from-loading-unwanted-kernel-modules

 

3rd Party:

• [Blogpost] Reduce Kubernetes Attack Surfaces https://blog.sonatype.com/kubesecops-kubernetes-security-practices-you-should-follow#:~:text=Reduce%20Kubernetes%20Attack%20Surfaces
• CIS Benchmark "CIS Distribution Independent Linux" https://www.cisecurity.org/benchmark/distribution_independent_linux/

Minimize IAM roles

 

3rd Party:

• [Wiki] Principle of least privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege
• [Common theory] Grant least privilege https://digitalguardian.com/blog/what-principle-least-privilege-polp-best-practice-information-security-and-compliance

Minimize external access to the network

• K8s quotas (restrict service.loadbalancer) https://kubernetes.io/docs/concepts/policy/resource-quotas/
• Admission control plugin: ResourceQuota https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/admission_control_resource_quota.md
• Restrict Access For LoadBalancer Service https://v1-17.docs.kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service

 

3rd Party:

• Host-Level firewall ufw (uncomplicated firewall) https://help.replicated.com/community/t/managing-firewalls-with-ufw-on-kubernetes/230
• ufw quick-start https://www.linode.com/docs/security/firewalls/configure-firewall-with-ufw/
• iptables cheat sheet https://gist.github.com/davydany/0ad377f6de3c70056d2bd0f1549e1017

Appropriately use kernel hardening tools such as AppArmor, seccomp

• Main doc (apparmor & k8s) https://kubernetes.io/docs/tutorials/clusters/apparmor/
• Main doc (seccomp & k8s) https://kubernetes.io/docs/tutorials/clusters/seccomp/

Minimize Microservice Vulnerabilities – 20%

Setup appropriate OS level security domains e.g. using PSP, OPA, security contexts

• PSP: https://kubernetes.io/docs/concepts/policy/pod-security-policy/
• Security Context: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
• OPA (Blog): https://kubernetes.io/blog/2019/08/06/opa-gatekeeper-policy-and-governance-for-kubernetes/

 

3rd Party:

• [Youtube] Intro to OPA https://www.youtube.com/watch?v=Yup1FUc2Qn0
• Kubernetes security context, security policy, and network policy – Kubernetes security guide (part 2). https://sysdig.com/blog/kubernetes-security-psp-network-policy/
• OPA: https://www.openpolicyagent.org/docs/latest/kubernetes-primer/
• OPA Admission Controller https://www.openpolicyagent.org/docs/v0.12.2/kubernetes-admission-control/

Manage Kubernetes secrets

• Main doc: https://kubernetes.io/docs/concepts/configuration/secret/
• Secret Encryption (etcd) https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
• Secret Encryption (KMS Provider) https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/

 

3rd Party:

• Kubernetes-Secrets-Store-CSI-Driver (used by 3rd-party secret stores such as Vault, KeyVault etc.) https://github.com/kubernetes-sigs/secrets-store-csi-driver
• Bitnami Sealed Secrets https://github.com/bitnami-labs/sealed-secrets
• Using secrets (Vault, Sealed), overview https://www.weave.works/blog/managing-secrets-in-kubernetes
• Demo for Vault integration: https://www.youtube.com/watch?v=IznsHhKL428&ab_channel=VMwareCloudNativeApps

Use container runtime sandboxes in multi-tenant environments (e.g. gvisor, kata containers)

• You can then use policies to enforce tenant isolation (Limit Ranges, Quotas, PSPs) https://kubernetes.io/docs/concepts/policy/
• You can use Pod anti-affinity to prevent Pods from different tenants from being scheduled on the same node. https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity

 

3rd Party:

• [GKE] Cluster multi-tenancy https://cloud.google.com/kubernetes-engine/docs/concepts/multitenancy-overview
• kata containers https://katacontainers.io/
  • Kata Containers, Docker and Kubernetes: How They All Fit Together https://platform9.com/blog/kata-containers-docker-and-kubernetes-how-they-all-fit-together/
  • How to use Kata Containers and CRI (containerd plugin) with Kubernetes https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md
• gVisor https://gvisor.dev/docs/
  • Step-by-Step gVisor https://thenewstack.io/how-to-implement-secure-containers-using-googles-gvisor/

Implement pod to pod encryption by use of mTLS

• Main doc: https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/

 

3rd Party:

• Istio: https://istio.io/latest/docs/tasks/security/authentication/authn-policy/#auto-mutual-tls
• Istio: https://developer.ibm.com/technologies/containers/tutorials/istio-security-mtls/
• Istio: https://www.istioworkshop.io/11-security/01-mtls/
• Mutual TLS Authentication (mTLS) De-Mystified https://codeburst.io/mutual-tls-authentication-mtls-de-mystified-11fa2a52e9cf

Supply Chain Security – 20%

Minimize base image footprint

 

3rd Party:

• [GCP] Kubernetes best practices: How and why to build small container images https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-how-and-why-to-build-small-container-images
• [GCP] Build the smallest image possible https://cloud.google.com/solutions/best-practices-for-building-containers#build-the-smallest-image-possible
• [GCP] Best practices https://cloud.google.com/blog/products/gcp/7-best-practices-for-building-containers
• "Distroless" Docker Images https://github.com/GoogleContainerTools/distroless
• https://hackernoon.com/tips-to-reduce-docker-image-sizes-876095da3b34

Secure your supply chain: whitelist allowed registries, sign and validate images

• Admission controllers https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#imagepolicywebhook
• One more link: https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/

 

3rd Party:

• OPA registry restriction: https://www.openpolicyagent.org/docs/latest/kubernetes-primer/
• Container Image Signatures in Kubernetes https://medium.com/sse-blog/container-image-signatures-in-kubernetes-19264ac5d8ce
• ImagePolicyWebhook (controller itself - custom): https://stackoverflow.com/questions/54463125/how-to-reject-docker-registries-in-kubernetes
  • ImagePolicyWebhook controller example: https://github.com/flavio/kube-image-bouncer
• Docker content trust https://docs.docker.com/engine/security/trust/
  • https://docs.docker.com/engine/reference/commandline/trust_sign/
  • https://docs.docker.com/engine/reference/commandline/trust_inspect/

Use static analysis of user workloads (e.g.Kubernetes resources, Docker files)

 

3rd Party:

• kubesec https://kubesec.io/
• CNCF kubehunter https://github.com/aquasecurity/kube-hunter
• [Online tool] kube-score https://kube-score.com/
• Kubernetes static code analysis with Checkov https://bridgecrew.io/blog/kubernetes-static-code-analysis-with-checkov/

Scan images for known vulnerabilities

 

3rd Party:

• clair https://github.com/quay/clair
• clair Quick Start https://quay.github.io/clair/howto/getting_started.html
• Scan Your Docker Images for Vulnerabilities https://medium.com/better-programming/scan-your-docker-images-for-vulnerabilities-81d37ae32cb3

Monitoring, Logging and Runtime Security – 20%

Perform behavioral analytics of syscall process and file activities at the host and container level to detect malicious activities

• Obsoleted https://v1-16.docs.kubernetes.io/docs/tasks/debug-application-cluster/falco/

 

3rd Party:

• Falco (CNCF): https://falco.org/
  • https://sysdig.com/blog/how-to-detect-kubernetes-vulnerability-cve-2019-11246-using-falco/
  • https://medium.com/@SkyscannerEng/kubernetes-security-monitoring-at-scale-with-sysdig-falco-a60cfdb0f67a

Detect threats within physical infrastructure, apps, networks, data, users and workloads

 

3rd Party:

• Guidance on Kubernetes Threat Modeling https://www.trendmicro.com/vinfo/us/security/news/virtualization-and-cloud/guidance-on-kubernetes-threat-modeling
• Threat matrix for Kubernetes https://www.microsoft.com/security/blog/2020/04/02/attack-matrix-kubernetes/

Detect all phases of attack regardless where it occurs and how it spreads

 

3rd Party:

• Just a concept: https://www.dnvgl.com/article/the-seven-phases-of-a-cyber-attack-118270
• Investigating Kubernetes Attack Scenarios in Threat Stack (part 1) https://www.threatstack.com/blog/kubernetes-attack-scenarios-part-1
• Investigating Kubernetes Attack Scenarios in Threat Stack (part 2) https://www.threatstack.com/blog/investigating-kubernetes-attack-scenarios-in-threat-stack-part-2
• Anatomy of a Kubernetes Attack - How Untrusted Docker Images Fail Us https://www.optiv.com/explore-optiv-insights/source-zero/anatomy-kubernetes-attack-how-untrusted-docker-images-fail-us

Perform deep analytical investigation and identification of bad actors within environment

 

3rd Party:

• Kubernetes Security 101: Risks and 29 Best Practices https://www.stackrox.com/post/2020/05/kubernetes-security-101/

Ensure immutability of containers at runtime

• "ReadOnlyRootFilesystem" (securityContext, PSP) https://kubernetes.io/docs/concepts/policy/pod-security-policy/#volumes-and-file-systems
• "readOnly" volume mount https://kubernetes.io/docs/concepts/storage/volumes/

 

3rd Party:

• Why I think we should all use immutable Docker images https://medium.com/sroze/why-i-think-we-should-all-use-immutable-docker-images-9f4fdcb5212f
• With immutable infrastructure, your systems can rise from the dead https://techbeacon.com/enterprise-it/immutable-infrastructure-your-systems-can-rise-dead
• Leveraging Kubernetes and OpenShift to Ensure that Containers are Immutable https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html/container_security_guide/keeping_containers_fresh_and_updateable#leveraging_kubernetes_and_openshift_to_ensure_that_containers_are_immutable

Use Audit Logs to monitor access

• Main doc: https://kubernetes.io/docs/tasks/debug-application-cluster/audit/

 

3rd Party:

• [Datadog, Step-by-Step] How to monitor Kubernetes audit logs https://www.datadoghq.com/blog/monitor-kubernetes-audit-logs/
• [Falco, Step-by-Step] Kubernetes Audit Logging https://docs.sysdig.com/en/kubernetes-audit-logging.html

Uncategorized and questions

• Restrict alpha and beta features https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restrict-access-to-alpha-or-beta-features - Solution
• etcd ACL https://www.programmersought.com/article/88121021471/
• prevent using node selectors (via PodNodeSelector Admission Controller) https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podnodeselector
• prevent kubelet from changing node labels (via NodeRestriction Admission Controller) https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#noderestriction
• Using Node Authorization (kubelet permissions) https://kubernetes.io/docs/reference/access-authn-authz/node/
• Multiple schedulers https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/
• Konnectivity https://kubernetes.io/docs/tasks/extend-kubernetes/setup-konnectivity/

Related links

• https://k21academy.com/docker-kubernetes/certified-kubernetes-security-specialist-cks-step-by-step-activity-guide-hands-on-lab/#
• https://deploy.live/blog/cks-certified-kubernetes-security-specialist-exam-preparation-guide/