Skip to content

Latest commit

 

History

History
57 lines (44 loc) · 4.94 KB

security.md

File metadata and controls

57 lines (44 loc) · 4.94 KB
Raw

Security

Security should be a chief concern for anyone running services online. This document provides an overview of the approach kaws uses for securing Kubernetes clusters and the various attack surfaces that users should be aware of. There are three primary classes of resources that need to be secured in the kaws system: the Kubernetes components (including etcd), the AWS EC2 servers running the cluster, and the SSL and SSH keys used to authenticate.

Kubernetes and etcd

Kubernetes itself is a complex system with several independent services that work in tandem to provide their benefits. The entire security profile of Kubernetes is out of scope for this document. The parts relevant to kaws are authentication and authorization between cluster administrators and the Kubernetes API and between the Kubernetes components themselves. kaws configures the cluster to use SSL client certificates for authentication in both cases. Three certificate authorities are created by kaws: one for etcd's client API, one for etcd's peer API (communication between etcd members), and one for Kubernetes. The Kubernetes master servers have a copy of the master certifiate/key pair and the node servers have a copy of the node certificate/key pair. At present, all non-master Kubernetes components share the same certificate and key. All Kubernetes servers have a certificate/key pair for flannel and the Kubernetes API server to use etcd's client API. The etcd servers have a copy of the private keys for both their client and server APIs. Each individual administrator has their own client certificate and key for the Kubernetes API. All of these certificates are signed by a certificate authority unique to the cluster. At this time, kaws itself does not perform any configuration related to authorization. If different administrators should have different levels of access to the Kubernetes API, this must be handled by the primary administrators.

Threat model

  • Compromised Kubernetes SSL credentials would give access to everything Kubernetes can see and control.

AWS resources

All servers in a kaws-built Kubernetes cluster exist in a public subnet of their VPC. This is necessary because each server needs to be accessed from outside the VPC: the bastion server must accept external SSH, the Kubernetes masters must serve the Kubernetes API, and the Kubernetes nodes must serve any web applications the administrators run on them. At this time, kaws does not use a VPN or private subnet with NAT, though this may change in future versions. Security groups are configured for each AWS resource to allow only the neccessary incoming traffic from the external Internet. The bastion server is the only server that accepts incoming SSH connections on port 22. The Kubernetes master servers (and load balancer) accept incoming connections on 443 only. The Kubernetes nodes accept incoming connections on both 80 and 443. Port 80 is open so that applications can redirect to a secure version on port 443 using HSTS headers. kaws does not enforce this in any way, however. SSH public keys are distributed to each server in the VPC via the --ssh-key option to kaws cluster init.

Threat model

  • Compromised SSH keys would give complete control of all data in the cluster.
  • Applications exposed to the external Internet are vulnerable to attacks that are beyond the scope of kaws, but could potentially result in anything up to and including unrestricted access to all data in the cluster.
  • Unencrypted communications to Kubernetes nodes on port 80 are vulnerable to a man in the middle attack, but can be mitigated by redirecting all HTTP requests to HTTPS and using HSTS.
  • SSL private keys are currently stored unencrypted on the EBS-backed EC2 instances themselves. Anyone with access to the servers has access to the private keys, which compromises the entire cluster.
  • The security of AWS itself is assumed, but any vulnerability in AWS's various services would apply to the Kubernetes cluster as well.

Public key infrastructure

One of the benefits of kaws is that it automates the creation of the public key infrastructure used to secure communications between Kubernetes components and administrators. kaws uses cfssl to generate certificates and keys and AWS Key Management Service to keep all private keys encrypted at rest.

Threat model

  • Compromised AWS KMS customer master keys would give an attacker the ability to decrypt to the cluster's private keys if they had access to the encrypted files, and potentially the entire etcd and/or Kubernetes APIs.
  • Vulnerabilities in cfssl and AWS KMS themselves affect any resources that rely on them for security.
  • Certificate signing requests generated by administrators are not verified, instead relying on the administrator's commit access to the kaws Git repository for authenticity. CSRs should be verified out of band if Git repository commit access alone is not suitable verification.