This repository has been archived by the owner on Mar 30, 2023. It is now read-only.
Add Azure Disk Encryption #73
Assignees
Comments
|
This would be a really good feature as our InfoSec team require us to enable encryption at rest. |
russcam
added a commit
that referenced
this issue
Jan 31, 2017
- Support both Key Encrypted Key and non- Key Encrypted Key modes - Introduce templates for master, client and data nodes to support the two encryption modes - Introduce vm-encrypted template that encrypts the OS disk TODO: - return BitLocker secrets for each encrypted VM - investigate if sequenceVersion can be hardcoded to 1 in the template Closes #73
russcam
added a commit
that referenced
this issue
Feb 7, 2017
- Support both Key Encrypted Key and non- Key Encrypted Key encryption modes - Introduce templates for master, client and data nodes to support the two encryption modes - Introduce encrypt-vm template that encrypts the OS disk and attached data disks For disk encryption to be applied: 1. The VM resource is created, with any attached data disks 2. The Encryption extension for the VM is created 3. The VM is updated with encryption settings from the Encryption extension NOTES: - There is no encryptionSettings on Data disks and specifying it returns a BadRequest. Looking at examples for Azure Disk Encryption, the encryptionSettings only get applied to the osDisk and the volumeType (OS, Data. All) determines which disks get encrypted. - The dependency chain is set up to deploy the VM disk encryption extensions for VMs after all VM resources are created, and to update VM storageProfiles after all disk encryption extensions have run TODO: - return BitLocker secret for each encrypted VM Closes #73
russcam
added a commit
that referenced
this issue
Feb 7, 2017
- Support both Key Encrypted Key and non- Key Encrypted Key encryption modes - Introduce templates for master, client and data nodes to support the two encryption modes - Introduce encrypt-vm template that encrypts the OS disk and attached data disks For disk encryption to be applied: 1. The VM resource is created, with any attached data disks 2. The Encryption extension for the VM is created 3. The VM is updated with encryption settings from the Encryption extension NOTES: - There is no encryptionSettings on Data disks and specifying it returns a BadRequest. Looking at examples for Azure Disk Encryption, the encryptionSettings only get applied to the osDisk and the volumeType (OS, Data. All) determines which disks get encrypted. - The dependency chain is set up to deploy the VM disk encryption extensions for VMs after all VM resources are created, and to update VM storageProfiles after all disk encryption extensions have run TODO: - return BitLocker secret for each encrypted VM Closes #73
|
Azure Disk Encryption has been started in branch https://github.com/elastic/azure-marketplace/tree/feature/encryption. I'm parking this branch for the moment as have not seen a successfully encrypted cluster; the ARM deployments generally succeed but the encryption operation, which, from the documentation, I understand is an async operation, fails. Here's an example extension.log file from In summary, it fails because it does not have permission to create directory Another example is in an issue that I opened on the azure linux extensions (Azure/azure-linux-extensions#334), deploying a 3 dedicated master, 3 data node cluster where each data node has 2 attached data disks in RAID 0. This one fails when running umount /oldroot (seen this issue often when deploying clusters with attached disks). The following is a PowerShell script that can be used to reproduce the issues happening. The general flow for encryption is:
Param(
[Parameter(Mandatory = $true,
HelpMessage="Name of the resource group to which the KeyVault belongs to. A new resource group with this name will be created if one doesn't exist")]
[ValidateNotNullOrEmpty()]
[string]$keyVaultResourceGroup,
[Parameter(Mandatory = $true,
HelpMessage="Name of the KeyVault in which encryption keys are to be placed. A new vault with this name will be created if one doesn't exist")]
[ValidateNotNullOrEmpty()]
[string]$keyVaultName,
[Parameter(Mandatory = $true,
HelpMessage="Location of the KeyVault. Important note: Make sure the KeyVault and VMs to be encrypted are in the same region / location.")]
[ValidateNotNullOrEmpty()]
[string]$location,
[Parameter(Mandatory = $true,
HelpMessage="Name of the AAD application that will be used to write secrets to KeyVault. A new application with this name will be created if one doesn't exist. If this app already exists, pass aadClientSecret parameter to the script")]
[ValidateNotNullOrEmpty()]
[string]$aadAppName,
[Parameter(Mandatory = $false,
HelpMessage="Client secret of the AAD application that was created earlier")]
[ValidateNotNullOrEmpty()]
[string]$aadClientSecret,
[Parameter(Mandatory = $false,
HelpMessage="Identifier of the Azure subscription to be used. Default subscription will be used if not specified.")]
[ValidateNotNullOrEmpty()]
[string]$subscriptionId,
[Parameter(Mandatory = $false,
HelpMessage="Name of optional key encryption key in KeyVault. A new key with this name will be created if one doesn't exist")]
[ValidateNotNullOrEmpty()]
[string]$keyEncryptionKeyName
)
function Write-Host-Dated($message, $foregroundColor) {
if (-not $foregroundColor) {
$foregroundColor = "White"
}
Write-Host "[$(Get-Date -format 'u')] $message" -ForegroundColor $foregroundColor
}
function PromptCustom($title, $optionValues, $optionDescriptions)
{
Write-Host $title
Write-Host
$a = @()
for($i = 0; $i -lt $optionValues.Length; $i++){
Write-Host "$($i+1))" $optionDescriptions[$i]
}
Write-Host
while($true)
{
Write-Host "Choose an option: "
$option = Read-Host
$option = $option -as [int]
if($option -ge 1 -and $option -le $optionValues.Length)
{
return $optionValues[$option-1]
}
}
}
function Prompt-Subscription() {
# Choose subscription. If there's only one we will choose automatically
$subs = Get-AzureRmSubscription
$subscriptionId = ""
if($subs.Length -eq 0) {
Write-Error "No subscriptions bound to this account."
return
}
if($subs.Length -eq 1) {
$subscriptionId = $subs[0].SubscriptionId
}
else {
$subscriptionChoices = @()
$subscriptionValues = @()
foreach($subscription in $subs) {
$subscriptionChoices += "$($subscription.SubscriptionName) ($($subscription.SubscriptionId))";
$subscriptionValues += ($subscription.SubscriptionId);
}
$subscriptionId = PromptCustom "Choose a subscription" $subscriptionValues $subscriptionChoices
}
return $subscriptionId
}
try {
if ($subscriptionId) {
Select-AzureRmSubscription -SubscriptionId $subscriptionId -ErrorAction Stop
}
else {
Write-Host "Please Login" -ForegroundColor Yellow
Login-AzureRmAccount -ErrorAction "Stop" 1> $null;
$subscriptionId = Prompt-Subscription
Select-AzureRmSubscription -SubscriptionId $subscriptionId
}
}
catch {
Write-Host "Please Login" -ForegroundColor Yellow
Login-AzureRmAccount -ErrorAction "Stop" 1> $null;
$subscriptionId = Prompt-Subscription
Select-AzureRmSubscription -SubscriptionId $subscriptionId
}
# elements from https://github.com/Azure/azure-powershell/blob/dev/src/ResourceManager/Compute/Commands.Compute/Extension/AzureDiskEncryption/Scripts/AzureDiskEncryptionPreRequisiteSetup.ps1
# Check if AAD app with $aadAppName was already created
$SvcPrincipals = (Get-AzureRmADServicePrincipal -SearchString $aadAppName);
if(-not $SvcPrincipals) {
# Create a new AD application if not created before
$identifierUri = [string]::Format("http://localhost:8080/{0}",[Guid]::NewGuid().ToString("N"));
$defaultHomePage = 'http://example.com';
$now = [System.DateTime]::Now;
$oneYearFromNow = $now.AddYears(1);
$aadClientSecret = [Guid]::NewGuid();
Write-Host-Dated "Creating new AAD application ($aadAppName) client secret ($aadClientSecret)";
$ADApp = New-AzureRmADApplication -DisplayName $aadAppName -HomePage $defaultHomePage -IdentifierUris $identifierUri -StartDate $now -EndDate $oneYearFromNow -Password $aadClientSecret;
$servicePrincipal = New-AzureRmADServicePrincipal -ApplicationId $ADApp.ApplicationId;
$SvcPrincipals = (Get-AzureRmADServicePrincipal -SearchString $aadAppName);
if(-not $SvcPrincipals) {
# AAD app wasn't created
Write-Error "Failed to create AAD app $aadAppName. Please log-in to Azure using Login-AzureRmAccount and try again";
return;
}
$aadClientID = $servicePrincipal.ApplicationId;
Write-Host-Dated "Created a new AAD Application ($aadAppName) with ID: $aadClientID ";
}
else {
if(-not $aadClientSecret) {
$aadClientSecret = Read-Host -Prompt "Aad application ($aadAppName) was already created, input corresponding aadClientSecret and hit ENTER. It can be retrieved from https://manage.windowsazure.com portal" ;
}
if(-not $aadClientSecret) {
Write-Error "Aad application ($aadAppName) was alerady created. Re-run the script by supplying aadClientSecret parameter with corresponding secret from https://manage.windowsazure.com portal";
return;
}
$aadClientID = $SvcPrincipals[0].ApplicationId;
}
# Create KeyVault
try {
$resGroup = Get-AzureRmResourceGroup -Name $keyVaultResourceGroup -ErrorAction SilentlyContinue;
}
catch [System.ArgumentException] {
Write-Host-Dated "Couldn't find resource group: ($keyVaultResourceGroup)";
$resGroup = $null;
}
#Create a new resource group if it doesn't exist
if (-not $resGroup) {
Write-Host-Dated "Creating new resource group: ($keyVaultResourceGroup)";
$resGroup = New-AzureRmResourceGroup -Name $keyVaultResourceGroup -Location $location;
Write-Host-Dated "Created a new resource group named $keyVaultResourceGroup to place keyVault";
}
try {
$keyVault = Get-AzureRmKeyVault -VaultName $keyVaultName -ErrorAction SilentlyContinue;
}
catch [System.ArgumentException] {
Write-Host-Dated "Couldn't find Key Vault: $keyVaultName";
$keyVault = $null;
}
#Create a new vault if vault doesn't exist
if (-not $keyVault) {
Write-Host-Dated "Creating new key vault: ($keyVaultName)";
$keyVault = New-AzureRmKeyVault -VaultName $keyVaultName -ResourceGroupName $keyVaultResourceGroup -Sku Standard -Location $location;
Write-Host-Dated "Created a new KeyVault named $keyVaultName to store encryption keys";
}
# Specify privileges to the vault for the AAD application - https://msdn.microsoft.com/en-us/library/mt603625.aspx
Set-AzureRmKeyVaultAccessPolicy -VaultName $keyVaultName -ServicePrincipalName $aadClientID -PermissionsToKeys wrapKey -PermissionsToSecrets get,set;
Set-AzureRmKeyVaultAccessPolicy -VaultName $keyVaultName -EnabledForDiskEncryption;
$diskEncryptionKeyVaultUrl = $keyVault.VaultUri;
$keyVaultResourceId = $keyVault.ResourceId;
if($keyEncryptionKeyName) {
try {
$kek = Get-AzureKeyVaultKey -VaultName $keyVaultName -Name $keyEncryptionKeyName -ErrorAction SilentlyContinue;
}
catch [Microsoft.Azure.KeyVault.KeyVaultClientException] {
Write-Host-Dated "Couldn't find key encryption key named : $keyEncryptionKeyName in Key Vault: $keyVaultName";
$kek = $null;
}
if(-not $kek) {
Write-Host-Dated "Creating new key encryption key named:$keyEncryptionKeyName in Key Vault: $keyVaultName";
$kek = Add-AzureKeyVaultKey -VaultName $keyVaultName -Name $keyEncryptionKeyName -Destination Software -ErrorAction SilentlyContinue;
Write-Host-Dated "Created key encryption key named:$keyEncryptionKeyName in Key Vault: $keyVaultName";
}
$keyEncryptionKeyUrl = $kek.Key.Kid;
}
# Create a new cluster
$templateVersion = "feature/encryption"
$elasticTemplate = "https://raw.githubusercontent.com/elastic/azure-marketplace/$templateVersion/src/mainTemplate.json"
$elasticResourceGroup = "encrypted-cluster"
$name = $elasticResourceGroup
$clusterParameters = @{
"artifactsBaseUrl"="https://raw.githubusercontent.com/elastic/azure-marketplace/$templateVersion/src"
"esVersion" = "5.1.2"
"esClusterName" = $name
"location" = "ResourceGroup"
"xpackPlugins" = "Yes"
"loadBalancerType" = "internal"
"vmDiskEncryption" = "Yes"
"activeDirectoryClientId" = $aadClientID
"activeDirectoryClientSecret" = $aadClientSecret
"keyEncryptionKey" = "Yes"
"keyEncryptionKeyUrl" = $keyEncryptionKeyUrl
"keyVaultName" = $keyVaultName
"keyVaultResourceGroup" = $keyVaultResourceGroup
"passphrase" = "Password123"
"vmDataDiskCount" = 0
"vmDataNodeCount" = 1
"dataNodesAreMasterEligible" = "Yes"
"adminUsername" = "russ"
"authenticationType" = "password"
"adminPassword" = "Password1234"
"securityAdminPassword" = "Password123"
"securityReadPassword" = "Password123"
"securityKibanaPassword" = "Password123"
}
Write-Host-Dated "Deploying cluster"
New-AzureRmResourceGroup -Name $elasticResourceGroup -Location $location
New-AzureRmResourceGroupDeployment -Name $name -ResourceGroupName $elasticResourceGroup -TemplateUri $elasticTemplate -TemplateParameterObject $clusterParameters
Write-Host-Dated "Deployed cluster" |
russcam
added a commit
that referenced
this issue
May 2, 2017
- Support both Key Encrypted Key and non- Key Encrypted Key encryption modes - Introduce templates for master, client and data nodes to support the two encryption modes - Introduce encrypt-vm template that encrypts the OS disk and attached data disks For disk encryption to be applied: 1. The VM resource is created, with any attached data disks 2. The Encryption extension for the VM is created 3. The VM is updated with encryption settings from the Encryption extension NOTES: - There is no encryptionSettings on Data disks and specifying it returns a BadRequest. Looking at examples for Azure Disk Encryption, the encryptionSettings only get applied to the osDisk and the volumeType (OS, Data. All) determines which disks get encrypted. - The dependency chain is set up to deploy the VM disk encryption extensions for VMs after all VM resources are created, and to update VM storageProfiles after all disk encryption extensions have run TODO: - return BitLocker secret for each encrypted VM Closes #73
|
Would it be possible to skip using the AzureDiskEncryptionForLinux extension entirely? i.e. simply generate a passphrase file, create the LUKs container, and mount? I've found other issues with the extension that make it seem very immature/incomplete:
There's no real benefit in using the extension, other than time saved rewriting code (and it's broken in Ubuntu anyway). |
|
We've parked this for the moment because, as you say, there seem to be several problems with using the Azure Disk Encryption VM Extension. Besides the points you've raised, one large blocker is the process flow of encryption with regards to applying it with any ARM template that deploys software to VMs; because encryption is an async process that finishes (successfully or not) after the ARM template has finished deployment, there is no callback mechanism or ability to schedule operations to happen after encryption is in place e.g. start Elasticsearch.
Another benefit to using Azure Disk Encryption over using DM-Crypt directly is key management with Azure Key Vault, but it's of little benefit if the implementation is broken 😄 Our current thinking for the way forward here is to use Managed Disks for encryption, which ties in with using VM Scale in #30 |
|
Now that the template uses managed disks (#178), which have encryption at rest enabled by default through Azure Storage Service Encryption (SSE), I'm going to close this issue as it's no longer relevant. |
|
This problem appears to have been caused by low memory. I had the minimum amount of RAM required (7gb) on the VM but still got the /oldroot errors. Try stopping the Elasticsearch service on the ES Data Node before running encryption on the OS and data disks. It worked for me. Example-- |
|
@darrell-tethr It's been a while since I was looking at disk encryption (February 2017) but the same issues were exhibited on Standard_DS2_v2 machines that meet the mininum 7GB RAM requirements. More crucially though, the Azure Disk Encryption implementation is not workable for an Azure Resource Manager template if there is a need for users/customers to have to SSH into machines to stop a process, run encryption, then start a process. The intention is that a customer/user can deploy an ARM template and have running software once the operation completes. A way that this could work with Azure Disk Encryption would be to allow the scheduling of scripts to run after encryption completes. Then it would be possible to
points 1 and 3 could be combined into point 3. |
|
@russcam Thanks for the feedback and clarification. I realize disk encryption can't be implemented directly through the ARM Templates. I was simply trying to manually encrypt the disks after deploying the Data Nodes. The only way I've been able to avoid the /oldroot errors and successfully encrypt the disks has been to shutdown the ES Process (JVM) in advance to conserve on RAM. It appears the JVM is consuming too much system RAM, even before any real data has been loaded. Note that I have not had any problems encrypting the dedicated Master node OS disk while the ES process is running. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Azure Disk Encryption is now generally available in all Azure public regions.
An Yes/No option could be provided in the template to allow disk encryption in the template.
The text was updated successfully, but these errors were encountered: