diff --git a/docs/tasks/administer-cluster/encrypt-data.md b/docs/tasks/administer-cluster/encrypt-data.md new file mode 100644 index 0000000000000..faa7384c0b568 --- /dev/null +++ b/docs/tasks/administer-cluster/encrypt-data.md @@ -0,0 +1,205 @@ +--- +title: Encrypting data at rest +redirect_from: +- "/docs/user-guide/federation/" +- "/docs/user-guide/federation/index.html" +- "/docs/concepts/cluster-administration/multiple-clusters/" +- "/docs/concepts/cluster-administration/multiple-clusters.html" +- "/docs/admin/multi-cluster/" +- "/docs/admin/multi-cluster.html" +--- + +{% capture overview %} +This page shows how to enable and configure encryption of secret data at rest. +{% endcapture %} + +{% capture prerequisites %} + +* {% include task-tutorial-prereqs.md %} + +* Kubernetes version 1.7.0 or later is required + +* Encryption at rest is alpha in 1.7.0 which means it may change without notice. Users may be required to decrypt their data prior to upgrading to 1.8.0. + +{% endcapture %} + +{% capture steps %} + +## Configuration and determining whether encryption at rest is already enabled + +The `kube-apiserver` process accepts an argument `--experimental-encryption-provider-config` +that controls how API data is encrypted in etcd. An example configuration +is provided below. + +## Understanding the encryption at rest configuration. + +```yaml +kind: EncryptionConfig +apiVersion: v1 +resources: + - resources: + - secrets + providers: + - identity: {} + - aesgcm: + keys: + - name: key1 + secret: c2VjcmV0IGlzIHNlY3VyZQ== + - name: key2 + secret: dGhpcyBpcyBwYXNzd29yZA== + - aescbc: + keys: + - name: key1 + secret: c2VjcmV0IGlzIHNlY3VyZQ== + - name: key2 + secret: dGhpcyBpcyBwYXNzd29yZA== + - secretbox: + keys: + - name: key1 + secret: YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY= +``` + +Each `resources` array item is a separate config and contains a complete configuration. The +`resources.resources` field is an array of Kubernetes resource names (`resource` or `resource.group`) +that should be encrypted. The `providers` array is an ordered list of the possible encryption +providers. Only one provider type may be specified per entry (`identity` or `aescbc` may be provided, +but not both in the same item). + +The first provider in the list will be used to encrypt resources going into storage. When reading +resources from storage the providers will be attempted in order to decrypt data. If no provider +can read the stored data, an error will be returned which will prevent clients from accessing that +resource. + +IMPORTANT: If any resource is not readable via the encryption config (because keys were changed), +the only recourse is to delete that key from the underlying etcd directly. Calls that attempt to +read that resource will fail until it is deleted or a valid decryption key is provided. + +### Providers: + +* `identity` results in the data being written as-is without encryption. + * When placed in the first position the resource will be decrypted as new values are written. +* `secretbox` uses XSalsa20 and Poly1305 to store data at rest + * It is fast, but a newer standard and may not be considered acceptable in environments that require high levels of review. + * It requires a 32 byte key. +* `aescbc` uses AES in CBC mode + * It is the recommended choice for encryption at rest but may be slightly slower than `secretbox`. + * It requires 32 byte keys +* `aesgcm` uses AES in GCM mode with a randomly assigned nonce + * This mode is the fastest, but is not recommended for use except when an automated key rotation scheme is implemented. * Because these nonces are small the secret key must be rotated frequently - at most every 200k writes. + * It supports 16, 24, or 32 byte keys, but 32 byte keys should always be used. + +Each provider supports multiple keys - the keys are tried in order for decryption, and if the provider +is the first provider the first key is used for decryption. + +## Encrypting your data + +Create a new encryption config file + +```yaml +kind: EncryptionConfig +apiVersion: v1 +resources: + - resources: + - secrets + providers: + - aescbc: + keys: + - name: key1 + secret: + - identity: {} +``` + +To create a new secret, generate a 32 byte random key and base64 encode it. On Linux or Mac OS X, the +following command will read 32 bytes of random data and then base 64 encode it. + +``` +head -c 32 /dev/urandom | base64 -i - -o - +``` + +Place that value in the secret field. + +Set the `--experimental-encryption-provider-config` flag on the `kube-apiserver` to point to the location +of the config file and restart your API server. + +IMPORTANT: Your config file contains keys that can decrypt the content in etcd and should be properly +permission restricted on your masters so that only the user that runs the `kube-apiserver` can read it. + + +## Verifying that data is encrypted + +Data is encrypted when written to etcd. After restarting your `kube-apiserver`, any newly created or +updated secret should be encrypted when stored. To check, you can use the `etcdctl` command line +program to retrieve the contents of your secret. + +Create a new secret called `secret1` in the `default` namespace: + +``` +kubectl create secret generic secret1 -n default --from-literal=mykey=mydata +``` + +Using the etcdctl commandline, read that secret out of etcd: + +``` +ETCDCTL_API=3 etcdctl get /kubernetes.io/secrets/default/secret1 [...] | hexdump -C +``` + +where `[...]` must be the additional arguments for connecting to the etcd server. The `hexdump` command +will format the encoded bytes into a readable form. Verify the resulting output is prefixed with +`k8s:enc:aescbc:v1:` which indicates the `aescbc` provider has encrypted the resulting data. + +Verify the secret is correctly decoded by retrieving the secret from the command line, and that the secret +contents match what was provided above (`mykey: mydata`). + +``` +kubectl describe secret generic -n default +``` + + +## Ensuring all secrets to be encrypted + +Since secrets are encrypted on write, performing an update on a secret will encrypt that content. + +``` +kubectl get secrets -o json | kubectl update -f - +``` + +Will read all secrets and then perform an update with encryption. If an error occurs due to a +conflicting write, retry the command. For larger clusters, you may wish to subdivide the secrets +by namespace or script an update. + + +## Rotating a decryption key + +Changing the secret without incurring downtime requires a multi step operation, especially in +the presence of a highly available deployment where multiple `kube-apiserver` processes are running. + +1. Generate a new key and add it as the second key entry for the current provider on all servers +2. Restart all `kube-apiserver` processes to ensure each server can decrypt using the new key +3. Make the new key the first entry in the `keys` array so that it is used for encryption in the config +4. Restart all `kube-apiserver` processes to ensure each server now encrypts using the new key +5. Run `kubectl get secrets -o json | kubectl update -f -` to update all secrets +6. Remove the old decryption key from the config after you back up etcd with the new key in use and update all secrets + +With a single `kube-apiserver`, step 2 may be skipped + + +## Decrypting all data + +To disable encryption at rest place the `identity` provider as the first entry in the config: + +```yaml +kind: EncryptionConfig +apiVersion: v1 +resources: + - resources: + - secrets + providers: + - identity: {} + - aescbc: + keys: + - name: key1 + secret: +``` + +and restart all `kube-apiserver` processes. Then run the command `kubectl get secrets -o json | kubectl update -f -` +to force all secrets to be decrypted. \ No newline at end of file diff --git a/docs/tasks/administer-cluster/securing-a-cluster.md b/docs/tasks/administer-cluster/securing-a-cluster.md new file mode 100644 index 0000000000000..e0be625c85cd5 --- /dev/null +++ b/docs/tasks/administer-cluster/securing-a-cluster.md @@ -0,0 +1,186 @@ +--- +assignees: +- smarterclayton +title: Securing a Cluster +redirect_from: +- "/docs/admin/cluster-security/" +- "/docs/admin/cluster-security.html" +- "/docs/concepts/cluster-administration/cluster-security/" +- "/docs/concepts/cluster-administration/cluster-security.html" +--- + +* TOC +{:toc} + +This document covers topics related to protecting a cluster from accidental or malicious access +and provides recommendations on overall security. + +## Controlling access to the Kubernetes API + +As Kubernetes is entirely API driven, controlling and limiting who can access the cluster and what actions +they are allowed to perform is the first line of defense. + +### Use Transport Level Security (TLS) for all API traffic + +Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the +majority of installation methods will allow the necessary certificates to be created and distributed to +the cluster components. Note that some components and installation methods may enable local ports over +HTTP and administrators should familiarize themselves with the settings of each component to identify +potentially unsecured traffic. + +### API Authentication + +When a cluster is installed, choose an appropriate authentication mechanism for the API servers to use that +match the common access patterns. For instance, small single user clusters may wish to use a simple certificate +or static Bearer token approach. Larger clusters may wish to integrate an existing or OIDC or LDAP server that +allow users to be subdivided into groups. + +All API clients must be authenticated, even those that are part of the infrastructure like nodes, +proxies, the scheduler, and volume plugins. These clients are typically [service accounts](/docs/admin/service-accounts-admin.md) and are created automatically at cluster startup. + +Consult the [authentication reference document](/docs/admin/authentication.md) for more information. + +### API Authorization + +Once authenticated, every API call is also expected to pass an authorization check. Kubernetes ships an +integrated Role-Based Access Control (RBAC) component that matches an incoming user or group to a +set of permissions bundled into roles. These permissions combine verbs (get, create, delete) with +resources (pods, services, nodes) and can be namespace or cluster scoped. A set of out of the box +roles are provided that offer reasonable default separation of responsibilty depending on what actions +a client might want to perform. + +As with authentication, simple and broad roles may be appropriate for smaller clusters, but as +more users interact with the cluster, it may become necessary to separate teams into separate +namespaces with more limited roles. + +With authorization, it is important to understand how updates on one object may cause actions in +other places. For instance, a user may not be able to create pods directly, but allowing them to +create a deployment (which creates pods on their behalf) will let them create those pods +indirectly. Likewise, deleting a node from the API will result in the pods scheduled to that node +being terminated and recreated on other nodes. The out of the box roles represent a compromise +between flexibility and the common use cases, but more limited roles should be carefully reviewed +to prevent accidental escalation. + +Consult the [authorization reference section](/docs/admin/authorization) for more information. + + +## Controlling the capabilities of a workload or user at runtime + +Authorization in Kubernetes is intentionally high level, focused on coarse actions on resources. +More powerful controls exist as **policies** to limit by use case how those objects act on the +cluster, themselves, and other resources. + +### Limiting resource usage on a cluster + +[Resource quota](/docs/concepts/policy/resource-quotas.md) limits the number or capacity of +resources granted to a namespace. This is most often used to limit the amount of CPU, memory, +or persistent disk a namespace can allocate, but can also control how many pods, services, or +volumes exist in each namespace. + +[Limit ranges](/docs/admin/limitrange) restrict the maximum or minimum size of some of the +resources above, to prevent users from requesting unreasonably high or low values for commonly +reserved resources like memory, or to provide default limits when none are specified. + + +### Controlling what privileges containers run with + +A pod definition contains a [security context](/docs/tasks/configure-pod-container/security-context.md) +that allows it to request access to running as a specific Linux user on a node (like root), +access to run privileged or access the host network, and other controls that would otherwise +allow it to run unfettered on a hosting node. [Pod security policies](/docs/concepts/policy/pod-security-policy.md) +can limit which users or service accounts can provide dangerous security context settings. + +In general most application workloads need limited access to host resources and so can run +successfully as a root process (uid 0) without access to host information. However, given +the privileges associated with the root user, it is always recommended that application +containers be written to run as a non-root user, and that administrators that wish to prevent +client applications from escaping their containers to use a restrictive pod security policy. + + +### Restricting network access + +The [network policy](/docs/tasks/administer-cluster/declare-network-policy.md) for a namespace +allows application authors to restrict which pods in other namespaces may access pods and ports +within their namespace. Many of the supported [Kubernetes networking providers](/docs/concepts/cluster-administration/networking.md) +now respect network policy. + +Quota and limit ranges can also be used to control whether users may request node ports or +load balanced services, which on many clusters can control whether those users applications +are visible outside of the cluster. + +Additional protections may be available that control network rules on a per plugin or per +environment basis, such as per-node firewalls, physically separating cluster nodes to +prevent cross talk, or advanced networking policy. + + +### Controlling which nodes pods may access + +By default there are no limits on which nodes a pod may run. Kubernetes offers a +[rich set of policies for controlling placement of pods onto nodes](/docs/concepts/configuration/assign-pod-node.md) +that are available to end users. For many clusters use of these policies to separate workloads +can be a convention that authors adopt or enforce via tooling. + +As an administrator, a beta admission plugin `PodNodeSelector` can be used to force pods +within a namespace to default or require a specific node selector, and if end users cannot +alter namespaces, this can strongly limit the placement of all of the pods in a specific workload. + + +## Protecting cluster components from compromise + +This section describes some common patterns for protecting clusters from compromise. + +### Restrict access to etcd + +Write access to the etcd backend for the API is equivalent to gaining root on the entire cluster, +and read access can be used to escalate fairly quickly. Administrators should always use strong +credentials from the API servers to their etcd server, such as mutual auth via TLS client certificates, +and it is often recommended to isolate the etcd servers behind a firewall that only the API servers +may access. It is not recommended to allow other components within the cluster to access the master +etcd instance unless proper security precautions are taken to limit access for those other components +to only keys the API does not use. + +### Enable audit logging + +The [audit logger](/docs/admin/audit/) is an alpha feature that records actions taken by the +API for later analysis in the event of a compromise. It is recommended to enable audit logging +and archive the audit file on a secure server. + +### Restrict access to alpha or beta features + +Alpha and beta Kubernetes features are in active development and may have limitations or bugs +that result in security vulnerabilities. Always assess the value an alpha or beta feature may +provide against the possible risk to your security posture. When in doubt, disable features you +do not use. + +### Rotate infrastructure credentials frequently + +### Review third party integrations before enabling them + +Many third party integrations to Kubernetes may alter the security profile of your cluster. When +enabling an integration, always review the permissions that extension requests before granting +it access. For example, many security integrations may request access to view all secrets on +your cluster which is effectively making that component a cluster admin. When in doubt, +restrict the integration to functioning in a single namespace if possible. + +Components that create pods may also be unexpectedly powerful if they can do so inside namespaces +like the `kube-system` namespace, because those pods can gain access to service account secrets +or run with elevated permissions if those service accounts are granted access to permissive +[pod security policies](/docs/concepts/policy/pod-security-policy.md) + +### Encrypt secrets at rest + +In general, the etcd database will contain any information accessible via the Kubernetes API +and may grant an attacker significant visibility into the state of your cluster. Always encrypt +your backups using a well reviewed backup and encryption solution, and consider using full disk +encryption where possible. + +Kubernetes 1.7 contains an alpha feature that will encrypt `Secret` resources in etcd, preventing +parties that gain access to your etcd backups from viewing the content of those secrets. While +this feature is currently experimental, it may offer an additional level of defence when backups +are not encrypted or an attacker gains read access to etcd. + +### Receiving alerts for security updates and reporting vulnerabilities + +Join the [kubernetes-announce](https://groups.google.com/forum/#!forum/kubernetes-announce) +group for emails about security announcements. See the [security reporting](/docs/reference/security.md) +page for more on how to report vulnerabilities. \ No newline at end of file