Registry API is now single point of failure for secure provider distribution

[conn]

Before #19389, Hashicorp Security's PGP key (32EA77F3) was distributed as part of the Terraform binary. This provided a consistent and local source of trust that remotely downloaded providers could be verified with. Now public keys are distributed alongside all other information required to download and verify their respective providers (see here for an example) and are not cached for subsequent downloads creating a "Trust on every use" anti-pattern for cryptographic verification.

This undoes a lot of advantages that cryptographic signatures provide for software distribution. A public key should be long-lived and exist on a system to verify many downloads. That way if the registry API were compromised, you'd still have a source of trust to fall back on to tell you that what Terraform is told to download is malicious. This would also help verify against Man-in-the-middle attacks that may happen while Terraform is downloading its providers. Additionally, the metadata provided by the registry API should also be signed as well which would be trivial as all values describing a provider release won't change.

For distributing public keys, please consider some solutions:

• caching and reusing the public key on first download, implementing the TOFU pattern
• distributing release keys as a separate package and also putting such keys in the GitHub repository for extra verification (Merkle trees are nice to use this way)
• creating an x509 structure where the root key is hard coded into Terraform and signed keys are distributed via the registry API

Terraform is a great tool and widely used on both a personal and corporate level. Supply chain attacks performed against Terraform users has the potential to cause a lot of harm since Terraform would generally be executed from a privileged environment which has access to secrets and/or permissions required to manage infrastructure. Please consider fixing this broken model a high priority.

Thanks,
JD

[conn]

For now, it's possible to verify and install providers by hand. First, install Hashicorp Security's public key located here:
gpg --search-keys 91A6E7F85D05C65630BEF18951852D87348FFC4C

Then download your needed providers, their checksums, and the signature for the checksums from https://releases.hashicorp.com/. For example:

curl -O https://releases.hashicorp.com/terraform-provider-aws/2.0.0/terraform-provider-aws_2.0.0_linux_amd64.zip
curl -O https://releases.hashicorp.com/terraform-provider-aws/2.0.0/terraform-provider-aws_2.0.0_SHA256SUMS
curl -O https://releases.hashicorp.com/terraform-provider-aws/2.0.0/terraform-provider-aws_2.0.0_SHA256SUMS.sig

You'll then want to verify the signature for the checksums:

gpg --verify terraform-provider-aws_2.0.0_SHA256SUMS.sig

You'll be looking for a line that says:

Good signature from "HashiCorp Security <security@hashicorp.com>"

The last thing you need to verify is the checksum itself:

sha256sum --ignore-missing --check terraform-provider-aws_2.0.0_SHA256SUMS

You'll be looking for output that looks like:

terraform-provider-aws_2.0.0_linux_amd64.zip: OK

Afterwards, feel free to unzip the provider binary directly into "$HOME/.terraform.d/plugins" and don't change the name as Terraform will use that to identify which version that provider is. More info on that here (essentially treat official providers as 3rd party plugins).

If you haven't set any particular version constraints then Terraform should properly discover and select the pre-installed provider. There's more background info to this process here.

If you're using Terraform from within a disposable environment such as a Docker container or EC2 instance, I would recommend baking your needed providers into your images.

[phinze]

Hi JD,

First of all, let me say thank you for the clear and thoughtful report here. You've come to us not only with a clear description of the issue at hand but also with some suggested solutions and a workaround as well. You've done excellent work here and this allows us to kick off a meaningful conversation right away.

What we're working towards overall is a model that allows both HashiCorp and other entities to publish providers in such a way that the lines of trust are clear for our users. Today, HashiCorp is still the only entity that publishes signed providers, as your research on the registry has shown.

The change you reference was meant as the first incremental step towards a fully realized model that includes considerations for the issues you raise, but you're absolutely right about the "trust on every use" problem in this current stage of the implementation.

After some discussion, we've decided the best next move is to add back in the hardcoded HashiCorp signing key for the 0.12 release. This will allow us and our users to focus on the many other changes in 0.12 without us having to dig into a change in the provider trust model.

We plan to switch over to the new scheme once we can release it alongside some additional controls and clear messaging about how the trust model is changing. All of the "consider some solutions" suggestions you made here are on the table for that phase, so thank you for those!

Let me know if this all makes sense from your side, and expect to see a PR to bake the signing key back in shortly.

Paul

[conn]

Solid answer! I could now be more please with how you've responded to this. Thank you for hearing both my concerns and suggestions, and acting on them accordingly. I'm excited to see the new model once it's available.

[conn]

Thanks for your work in merging that.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.