AG needs PKCS12 format but Terraform only provides PEM support

[holderbaum]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

To configure the Application Gateway Resource with a TLS certificate, a pkcs12 file format is required. Unfortunately, all other native terraform resources (outside azure) work with traditional PEM format.

We currently try to provision an AG with a certificate that has been generated using the ACME terraform provider in conjunction with its Azure DNS provider (https://www.terraform.io/docs/providers/acme/dns_providers/azure.html).

The ACME certificate resource (https://www.terraform.io/docs/providers/acme/r/certificate.html) creates a Lets Encrypt certificate successfully and returns its public and private key as PEM.

Do get this into the AG we currently use an external datasource that calls openssl:

Terraform

data "external" "certificate-pfx" {
  program = ["bash", "${path.module}/data-sources/generate-pfx.sh"]

  query = {
    certificate_pem = "${acme_certificate.certificate.certificate_pem}"
    private_key_pem = "${acme_certificate.certificate.private_key_pem}"
  }
}

resource "azurerm_application_gateway" "ag" {
  # [...]

  http_listener {
    # [...]
    ssl_certificate_name = "ag-ssl-certificate"
  }

  ssl_certificate {
    name     = "ag-ssl-certificate"
    data     = "${data.external.certificate-pfx.result["pfx"]}"
    password = ""
  }

Data Source

#!/bin/bash
set -eu -o pipefail

export certificate_pem
export private_key_pem

eval "$(jq -r '@sh "certificate_pem=\(.certificate_pem) private_key_pem=\(.private_key_pem)"')"

cert_file="$(mktemp -p .)"
trap 'rm -f $cert_file' EXIT

echo "${certificate_pem}${private_key_pem}" >"$cert_file"

pfx="$(openssl pkcs12 \
  -in "$cert_file" \
  -export \
  -password "pass:" \
  |base64 -)"

jq -n --arg pfx "$pfx" '{"pfx":$pfx}'

Actual Issue

While this works from a technical perspective, it is far from optimal. The Data Source is executed on every terraform apply and the resulting pkcs12 archive is different on every execution, at least on the binary level - even though the inner PEM files stay the same

That means, for every apply the AG "changes" it's certificate which takes roughly 10 minutes.

Do you have any smart ideas on how to make this work better?

Ideally, the azure resources should use the same certificate formats as all other TF resources, that would be PEM.

New or Affected Resource(s)

• azurerm_application_gateway

Thank you very much :)

[dominik-lekse]

We are having a similar issue.

I suggest the following solution to solve the problem: We could implement a pkcs12 data source in the Terraform TLS provider which takes PEM encoded certificates and private keys and outputs them as PKCS12. This allows to reuse this functionality. For example, this is also relevant for Azure Key Vault certificates as they only support PKCS12.

[draggeta]

This may be just me, but wouldn't it make more sense to have the ACME provider be able to provide multiple formats? Either way, I'm also interested in this solution.

[Leon99]

@dominik-lekse Re.

this is also relevant for Azure Key Vault certificates as they only support PKCS12.

According to https://www.terraform.io/docs/providers/azurerm/r/key_vault_certificate.html#content_type, it does support PEM.

[tombuildsstuff]

hey @holderbaum @dominik-lekse @draggeta @Leon99

Thanks for opening this issue :)

Taking a quick look into this whilst it's unfortunate that some of the Azure API's only accept PFX files I'd agree with @dominik-lekse that this probably belongs in the TLS Provider. Whilst we could potentially add support for this to the ACME Provider I don't feel like it's a problem specific to that Provider (since being able to convert between certificate types could be useful in other contexts too) - as such I'm going to transfer this issue over to the TLS Provider.

Thanks!

[zimbatm]

this issue is a duplicate of #29

[MattMencel]

@holderbaum Thanks for your code above. I've been experimenting with this, just putting the content in a local file resource, and you may be able to work around the PKCS12 changing every time by using the lifecycle ignore_changes feature.

resource "local_file" "pk12" {
  content  = "${module.agw_cert.certificate_pfx}"
  filename = "${path.module}/cert.pfx"

  lifecycle {
    ignore_changes = ["content"]
  }
}

The PKCS12 content gets re-generated every time, but it doesn't actually update the resource. If you ever wanted to update it for real you could taint the resource or remove the lifecycle block.

It's not great, but it may work short term until the TLS provider adds the PKCS12 export option.

[apparentlymart]

To me this feels similar to other situations in Terraform where we've established a standard way of representing a particular type of data in Terraform configuration and expected providers to convert to the upstream format they need, since that allows different systems to compose better.

This is the same as convention that timestamps should always be in the RFC 3339 format, and providers are responsible for converting from that to whatever timestamp format the remote API expects, so that Terraform configurations do not need to be littered with conversion functions, and any resource type that produces a timestamp only needs to produce one format.

In other words, I think it's better to retain the convention that private keys and certificates in Terraform are always in PEM format, and that any provider interfacing with an upstream system that doesn't support PEM must itself handle the conversion, accepting PEM certificates as input and producing PEM certificates as output. By standardizing on one format, we know that everything can compose together without needing to worry about format selections.

Since a PEM certificate contains only the certificate data, with the private key supplied out-of-band, this means also that providers accepting certificates and their associated private keys will need to have two separate arguments, rather than packaging them together into one. This composes better with tls_private_key and tls_certificate (and similar) anyway, since they consider these two artifacts to be separate. It also means that the provider can mark the private key as sensitive while leaving the certificate plain, allowing it to be seen in plan output.

Note also that the Terraform language only supports Unicode strings, so using binary formats like PKCS12 would require base64-encoding them. In Terraform 0.11 binary data tends to just get silently corrupted in certain cases because the unicode-only requirement is not consistently enforced. In 0.12 this will be checked explicitly, producing errors on any attempt to process binary data in strings rather than silently corrupting it.

PEM certificates, since they are ASCII-only, can be used directly inside strings and converted to a binary format like PKCS12 just in time to be sent to the API in the target provider.

[asc-adean]

Would this resolve the use case of uploading certificates (in any valid format) into an Azure Key Vault and utilizing them as a valid data source for maintaining Application Gateway SSL certificates? Since Azure itself does not support this natively (https://feedback.azure.com/forums/217313-networking/suggestions/31089529-support-ssl-certificates-stored-in-key-vault-secre) this would be a great way to handle multiple gateways with identical keys being updated via automation.

[flokli]

While I agree with @apparentlymart that in general, terraform providers should accept the "standard" format (in this case, PEM) and convert internally if the upstream service provider expects something else, sometimes this isn't sufficient - imagine receiving PEM from a terraform module, but having to store this somewhere else (s3, consul, local file), while the service picking it up only understands PKCS12.

Instead of everybody implementing his own workarounds by manually calling openssl, terraform-provider-tls should provide some tooling IMHO.

tls_public_key

It already allows calculating public keys by passing in a private key, getting various public key representations (OpenSSH format), and fingerprints.

For example, to get the OpenSSH pubkey of a Private Key provided in PEM format

data "tls_public_key" "example" {
  private_key_pem = "${file("~/.ssh/id_rsa")}"
}
output "openssh_pubkey" {
  value = "${tls_public_key.example}";
}

tls_certificate

I don't see much against adding a tls_certificate resource to juggle with certificates and chains.

Possible arguments

• private_key_pem
• public_key_pem
• pkcs12
• pkcs12_password (Optional, only valid with pkcs12 being set)

(With pkcs12 and (private,public)_key_pem being mutually exclusive)

Exported attributes

• private_key_pem
• public_key_pem
• fullchain_pem
• pkcs12 (base64 encoded)
• fingerprint_…

This could be used to add support for converting from PEM to PKCS12 and back in a very unobtrusive manner.

Example

Generate the Openssh Public Key from a PKCS12 file:

# load the pkcs12 file
data "tls_certificate" "as_pkcs12" {
  pkcs12 = "${file("foo.pfx")}"
}
# use the private key from the pkcs12 file
data "tls_public_key" "as_pem" {
  private_key_pem = "${tls_certificate.as_pkcs12.private_key_pem}"
}
# output the openssh representation of the corresponding public key
output "foo" {
  value = "${tls_public_key.as_pem.public_key_openssh}"
}

[apparentlymart]

Exporting other formats from the tls provider resources (as we do with the alternate _openssh format for public keys) seems like a reasonable compromise for exporting data to "generic" data stores like S3.

I would like to keep PEM only for inputs to the resources to reinforce that it is the conventional format, and I'd ask that we also implement support for PEM as input on azurerm_application_gateway first lest supporting PKCS12 in outputs in this provider become a crutch for that and the requirement for a plethora of different export formats then infecting other resource types (in other providers) that may also generate certificates.

[identifysun]

Actually, We can reference to ACME provider, The terraform resource support
certificate_p12_password argument and certificate_p12 attribute.

The reference links as follow:

https://www.terraform.io/docs/providers/acme/r/certificate.html

[mhaarbrink]

Has there been any progress on the TLS provider adding tooling to support converting the PEM's to PKCS12 or more clean workarounds for local code to do the conversations outside of terraform?

I understand what @apparentlymart is saying, but Microsoft has been asked about support of PEM for their application gateways, app services, key vaults and so on and it's not on the roadmap. Microsoft has been using PKCS12 longer than Hashicorp/Terraform have been around so it's not simple or likely change I think. Additionally, I don't think avoiding this functionality in the TLS provider encourages Microsoft to adopt the PEM format. If anything it encourages Azure users to not use Terraform for solutions related to certificates.

I did look into using the ACME provider but our customer is using Venafi so we already have the PEM and need to convert it. We don't need a separate tool to generate the cert.

@apparentlymart I saw you mentioned something about PKCS12 being binary and Terraform being Unicode. Is that an actual hard deal breaker like it's literally impossible for the TLS provider to offer this functionality? Or are you just saying it introduces some challenges or complexity?

[chilicat]

I have created a provider that helps to create PKCS12 from PEM files:

https://registry.terraform.io/providers/chilicat/pkcs12/latest/docs/resources/from_pem

[marcmarcet]

This is the workaround I came up with using null resource, in my case for converting cloudflare origin certs (pem) to pfx:

# writes the cert to a temp file.
resource "local_file" "certificate_pem" {
    content     = cloudflare_origin_ca_certificate.this.certificate
    filename = "${path.root}/tmp/certificate"
}

# writes the private key to a temp file.
resource "local_file" "private_key_pem" {
    content     = tls_private_key.this.private_key_pem
    filename = "${path.root}/tmp/certificate.key"
}

# uses both files to generate the pfx with openssl
resource "null_resource" "pem2pfx" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "openssl pkcs12 -export -in ${local_file.certificate_pem.filename} -inkey ${local_file.private_key_pem.filename} -out ${path.root}/tmp/certificate.pfx -passout pass:PASSWORD"
  }
}

# reads the pfx
data "local_file" "certificate_pfx" {
  filename = "${path.root}/tmp/certificate.pfx"

  depends_on = [
    null_resource.pem2pfx
  ]
}

and then you can reference it like this:

data.local_file.certificate_pfx.content_base64

[ankit-kumar-mck]

@marcmarcet

pkcs12_from_pem.my_pkcs12: Creating...
local_file.certificate_pem: Creation complete after 0s [id=2220ae52ac00701c8386f9740d477741bdcc6ee1]
null_resource.pem2pfx: Creating...
null_resource.pem2pfx: Provisioning with 'local-exec'...
null_resource.pem2pfx (local-exec): Executing: ["/bin/sh" "-c" "openssl pkcs12 -export -in ./tmp/fullchain.pem -inkey ./tmp/privkey.pem -out ./tmp/certificate.pfx -passout pass:mypassword"]
pkcs12_from_pem.my_pkcs12: Creation complete after 0s [id=648a92f8f83536350d12ffdcd257a048f599da98]
null_resource.pem2pfx (local-exec): No certificate matches private key


Error: Error running command 'openssl pkcs12 -export -in ./tmp/fullchain.pem -inkey ./tmp/privkey.pem -out ./tmp/certificate.pfx -passout pass:mypassword': exit status 1. Output: No certificate matches private key

[Socolin]

I would like to have this too.

My use case is to import import certificates from azure keyvault, stored as pkc12 in kubernetes cluster as secret for nginx ingress (cert/key as pem).

[detro]

Superseded by #205.

[cicorias]

TBH, really confusing set of threads here -- while all these are closed -- i don't see the PFX/PKCS in the Terraform TLS provider. Is it somewhere else?

[jbg]

@cicorias the superseding issue is linked to in the comment immediately before yours, and is still open.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.