Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding auditing docs #2431

Merged
merged 1 commit into from
Sep 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions site/content/docs/contributing/1.0-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,6 @@ To reach "GA" [grade][deprecation-policy] kind needs to at minimum:
- [x] Support non-AMD64 architectures (namely ARM) - [#166]
- [ ] Automated publishing of Kubernetes release based kind "node" images - [#197]
- [x] Support for runtimes other than docker/default including podman, ignite etc.
- [ ] Enable audit-logging
- TODO: should this be post-GA? this could probably be an extension
- [x] First class support for skewed node (Kubernetes) versions (I believe this is relatively first-class now, things should work fine if you specify different node images)
- ... TBD, more will be added here ...

Expand Down
129 changes: 129 additions & 0 deletions site/content/docs/user/auditing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
---
title: "Auditing"
menu:
main:
parent: "user"
identifier: "user-auditing"
weight: 4
description: |-
This guide covers how to enable Kubernetes API [auditing] on a kind cluster.

[auditing]: https://kubernetes.io/docs/tasks/debug-application-cluster/audit/
---

## Overview

Kubernetes auditing provides a security-relevant, chronological set of records documenting the sequence of actions in a cluster. Auditing requires a file to define the [audit policy] and a backend configuration to store the logged events. Auditing supports two types of backends: log (file) & webhook. The following exercise uses the log backend.

Steps:

- Create the local audit-policy file
- Mount the local audit-policy file into the kind control plane
- Expose the control plane mounts to the API server
- Enable the auditing API flags
- Create a cluster

## Setup

### Create an `audit-policy.yaml` file

The [audit policy] defines the level of granularity outputted by the Kubernetes API server. The example below logs all requests at the "Metadata" level. See the [audit policy] docs for more examples.

{{< codeFromInline lang="bash" >}}
cat <<EOF > audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
EOF
{{< /codeFromInline >}}

### Create a `kind-config.yaml` file.

To enable audit logging, use kind's [configuration file] to pass additional setup instructions. Kind uses `kubeadm` to provision the cluster and the configuration file has the ability to pass `kubeadmConfigPatches` for further customization.

{{< codeFromInline lang="bash" >}}
cat <<EOF > kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
# enable auditing flags on the API server
extraArgs:
audit-log-path: /var/log/kubernetes/kube-apiserver-audit.log
audit-policy-file: /etc/kubernetes/policies/audit-policy.yaml
# mount new files / directories on the control plane
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these things always cause confusion, maybe we should add something more specific indicating "this happens in the kind node" and in extraMounts "this happens in your host"

extraVolumes:
- name: audit-policies
hostPath: /etc/kubernetes/policies
mountPath: /etc/kubernetes/policies
readOnly: true
pathType: "DirectoryOrCreate"
- name: "audit-logs"
hostPath: "/var/log/kubernetes"
mountPath: "/var/log/kubernetes"
readOnly: false
pathType: DirectoryOrCreate
Comment on lines +61 to +70
Copy link

@meowrison meowrison Sep 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of a nit: There is inconsistent value definition conventions here, some quoted some not. This may be confusing to some. Awesome to see this overall!

# mount the local file on the control plane
extraMounts:
- hostPath: ./audit-policy.yaml
containerPath: /etc/kubernetes/policies/audit-policy.yaml
readOnly: true
EOF
{{< /codeFromInline >}}

## Launch a new cluster

{{< codeFromInline lang="bash" >}}
kind create cluster --config kind-config.yaml
{{< /codeFromInline >}}

## View audit logs

Once the cluster is running, view the log files on the control plane in `/var/log/kubernetes/kube-apiserver-audit.log`.

{{< codeFromInline lang="bash" >}}
docker exec kind-control-plane cat /var/log/kubernetes/kube-apiserver-audit.log
{{< /codeFromInline >}}

## Troubleshooting
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice


If logs are not present, let's ensure a few things are in place.

### Is the local audit-policy file mounted in the control-plane?

{{< codeFromInline lang="bash" >}}
docker exec kind-control-plane ls /etc/kubernetes/policies
{{< /codeFromInline >}}

Expected output:

```bash
audit-policy.yaml
```

### Does the API server contain the mounts and arguments?

{{< codeFromInline lang="bash" >}}
docker exec kind-control-plane cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep audit
{{< /codeFromInline >}}

Expected output:

```bash
- --audit-log-path=/var/log/kubernetes/kube-apiserver-audit.log
- --audit-policy-file=/etc/kubernetes/policies/audit-policy.yaml
name: audit-logs
name: audit-policies
name: audit-logs
name: audit-policies
```

If the control plane requires further debugging use `docker exec -it kind-control-plane bash` to start an interactive terminal session with the container.

[audit policy]: https://kubernetes.io/docs/tasks/debug-application-cluster/audit/#audit-policy
[configuration file]: /docs/user/configuration