stacks: An early minimal implementation of ephemeral values

[apparentlymart]

This PR is does some plumbing similar to #35273, but for the stack language runtime instead of the module language runtime.

This part does not have an explicit experiment opt-in as we did for the modules language, because the stacks language as a whole remains experimental anyway.

If stacks gets stabilized before ephemeral values then the ephemeral values features in the stacks language would be stabilized but ephemeral values from a stack config could only be passed into a component whose root module is participating in the ephemeral_values language experiment, because a variable declared in a non-experiment-opted module cannot declare ephemeral = true. If we decide that we don't want to stabilize the stacks language parts of ephemeral values along with stacks then we can potentially drop this by just disabling the handling in the stackconfig package, making the other parts of this effectively dead code until we're ready to revise them.

---

When used as part of HCP Terraform, provider authentication for stacks is handled by having HCP Terraform generate a JSON Web Token and pass it into the topmost stack configuration as an input variable.

However, currently input variables are not allowed to vary between the plan and apply phases and so HCP Terraform employs a temporary hack: it writes the JSON Web Token to disk in a file whose path stays consistent between plan and apply and then passes the file path as the input variable value, with the expectation that the stack configuration will use the file function to read its content and assign it to an argument inside the appropriate provider block.

This relies on the fact that provider configurations are effectively ephemeral, despite us not previously using that terminology to describe them, and so Terraform has no problem with the content of that file changing between plan and apply. However, the new concept of ephemeral values aims to turn that concept of ephemerality into a cross-cutting concern that can be used without such workarounds. Ephemeral input variables make it explicit that the value is expected to change between plan and apply, and then Terraform Core ensures that the resulting value can only be used in locations where that ephemerality can be supported

The primary goal of this PR, then, is to allow changing how HCP Terraform handles JSON Web Tokens so that they can pass into the stacks runtime directly in the Terraform Core RPC API requests, without having to be written to disk.

The secondary goal (and the reason for doing this now, rather than later) is to give us another opportunity to get some practical experience using ephemeral values in alpha releases throughout the Terraform v1.10 development period, so that if there are any as-yet-unknown design problems then we're more likely to learn about them sooner.

This enables a stacks-language-flavored equivalent of the example I wrote for #35273:

required_providers {
  aws = {
    source = "hashicorp/aws"
  }
}

variable "aws_jwt" {
  type      = string
  ephemeral = true
  sensitive = true
}

locals {
  aws_read_role_arn  = "arn:aws:iam::XXXXX:role/terraform-read"
  aws_write_role_arn = "arn:aws:iam::XXXXX:role/terraform-full"
}

provider "aws" "main" {
  config {
    region = "us-west-2"

    assume_role_with_web_identity {
      role_arn           = terraform.applying ? local.aws_write_role_arn : local.aws_read_role_arn
      web_identity_token = var.aws_jwt
    }
  }
}

If this is accepted then the HCP Terraform part of this, implemented elsewhere, would include:

• Changing the HCP Terraform stack deployment language implementation so that identity tokens are given directly as values in memory rather than indirectly through files on disk, using identity_token.NAME.jwt instead of identity_token.NAME.jwt_filename.
• Using the newly-added input variable metadata to validate that identity tokens can be used only to define input variables that are marked as ephemeral.

In practice I expect we'd retain the option of writing the files to disk for at least a little while to avoid the need to update every existing stack deployment configuration example at once. However, we should aim to avoid writing the files to disk unless the filename attribute in particular is requested, so that those using the new approach would actually benefit from not having their tokens hit disk during an operation, and ideally phase out the old workaround entirely (or, at least, mark it as explicitly deprecated) before the deployment configuration language becomes subject to any compatibility promises.

[liamcervante]

Looks good to me!

[github-actions]

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.