| title | titleSuffix | description | services | ms.topic | ms.date |
|---|---|---|---|---|---|
Secure secrets with Key Vault |
Azure Kubernetes Service |
Learn how to get Secrets from Azure Key Vault in Azure Kubernetes Service (AKS) using AAD Pod Identity and Secrets Store CSI provider |
container-service |
tutorial |
05/03/2020 |
A Secret in Kubernetes is meant to save sensitive and secure data like passwords, certificates, and keys. These Secret objects aren't secret or secure! They're encoded using Base64 and saved into etcd. Also, by default, any pod have access to them.
To secure these secrets, one solution could be to encrypt data at rest in Kubernetes as explained in Kubernetes documentation. But this feature (KMS) is not yet supported in AKS. However some work is in progress. Check the AKS Roadmap and the KMS Plugin for Key Vault.
Another solution would be using Azure Key Vault. It can securely encrypt sensitive data like keys, secrets, files, and certificates. Even more than just encryption, they offer powerful features like key rotation, expiration date and access policies.
Now, to access Key Vault, a password is needed. Thus it will be non-securely saved in a Secret object in etcd! We returned back to the first problem. Fortunately in Azure, AKS can connect to Key Vault or SQL Database… using an Identity. This connection could be done using the open-source project Pod Identity. It uses Azure AD to create an Identity and assign the roles and resources.
Now that we have access to Key Vault, we can use its SDK or REST API in the application to retrieve the secrets. The SDK has support for .NET, Java, Python, JS, Ruby, PHP, etc. Or we can retrieve the secrets from a mounted volume. Historically, in Azure, this solution was implemented through Kubernetes Key Vault Flex Volume. Now it's being deprecated. The new solution is Azure Key Vault provider for Secret Store CSI driver. Which is the Azure implementation of Secrets Store CSI driver.
This tutorial will help you to securely retrieve secrets in Key Vault right from the Pod using Secrets Store CSI and AAD Pod Identity.
To complete this workshop, we need az, kubectl and helm CLI. Also, we need to create the following resources in Azure: • AKS cluster that uses Service Principal or Managed Identity through the variable $isAKSWithManagedIdentity. • Key Vault with the following secrets: “DatabaseLogin: DbUserName” and “DatabasePassword: MyP@ssword123456”. • Container Registry (ACR). We provision these resources from the Azure Portal or using the following Powershell script:
$suffix = "demo01"
$subscriptionId = (az account show | ConvertFrom-Json).id
$tenantId = (az account show | ConvertFrom-Json).tenantId
$location = "westeurope"
$resourceGroupName = "rg-" + $suffix
$aksName = "aks-" + $suffix
$aksVersion = "1.16.13"
$keyVaultName = "keyvaultaks" + $suffix
$secret1Name = "DatabaseLogin"
$secret2Name = "DatabasePassword"
$secret1Alias = "DATABASE_LOGIN"
$secret2Alias = "DATABASE_PASSWORD"
$identityName = "identity-aks-kv"
$identitySelector = "azure-kv"
$secretProviderClassName = "secret-provider-kv"
$acrName = "acrforaks" + $suffix
$isAKSWithManagedIdentity = "true"
# echo "Creating Resource Group..."
$resourceGroup = az group create -n $resourceGroupName -l $location | ConvertFrom-Json
# echo "Creating ACR..."
$acr = az acr create --resource-group $resourceGroupName --name $acrName --sku Basic | ConvertFrom-Json
az acr login -n $acrName --expose-token
If ($isAKSWithManagedIdentity -eq "true") {
echo "Creating AKS cluster with Managed Identity..."
$aks = az aks create -n $aksName -g $resourceGroupName --kubernetes-version $aksVersion --node-count 1 --attach-acr $acrName --enable-managed-identity | ConvertFrom-Json
} Else {
echo "Creating AKS cluster with Service Principal..."
$aks = az aks create -n $aksName -g $resourceGroupName --kubernetes-version $aksVersion --node-count 1 --attach-acr $acrName | ConvertFrom-Json
}
# retrieve the existing or created AKS
$aks = (az aks show -n $aksName -g $resourceGroupName | ConvertFrom-Json)
# echo "Connecting/authenticating to AKS..."
az aks get-credentials -n $aksName -g $resourceGroupName
echo "Creating Key Vault..."
$keyVault = az keyvault create -n $keyVaultName -g $resourceGroupName -l $location --enable-soft-delete true --retention-days 7 | ConvertFrom-Json
# $keyVault = az keyvault show -n $keyVaultName | ConvertFrom-Json # retrieve existing KV
echo "Creating Secrets in Key Vault..."
az keyvault secret set --name $secret1Name --value "DbUserName" --vault-name $keyVaultName
az keyvault secret set --name $secret2Name --value "P@ssword123456" --vault-name $keyVaultName
Important
Key Vault, AKS and Identity are in the same resource group here for simplicity. But they can be deployed on different ones.
We’ll start by installing Secrets Store CSI driver using Helm charts into a separate namespace.
helm repo add csi-secrets-store-provider-azure https://raw.githubusercontent.com/Azure/secrets-store-csi-driver-provider-azure/master/charts
"secrets-store-csi-driver-provider-azure" has been added to your repositories
kubectl create ns csi-driver
namespace/csi-driver created
helm install csi-azure csi-secrets-store-provider-azure/csi-secrets-store-provider-azure --namespace csi-driver
NAME: csi-azure
LAST DEPLOYED: Thu Jun 11 11:57:28 2020
NAMESPACE: csi-driver
STATUS: deployed
REVISION: 1
TEST SUITE: None
Let's check the new created pods:
kubectl get pods --namespace=csi-driver
NAME READY STATUS RESTARTS AGE
csi-azure-csi-secrets-store-provider-azure-9mf84 0/1 ContainerCreating 0 3s
csi-azure-secrets-store-csi-driver-rpn7f 0/3 ContainerCreating 0 3s