Add attributes for gcs backend to specify an existing bucket in another project

[Reifier]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Fairly simple scenario. I have tf configuration which I am going to use to create a project B from scratch. However, I want to store the state of that new project and all config in a gcs bucket in project A which have existed beforehand.

terraform {
  backend "gcs" {
    bucket  = "I-want-this-state-in-A"
    prefix  = "terraform/state"
  }
}

It seems like once upon the time backend config for gcs could include project and region. It seems to no longer be the case.

The project data seems to be fetched from provider block that looks like so:

provider "google" {
  region  = myregionB
  project = myprojectB
}

Now this is a problem, because all of my terraform resources use this config to create things in B including B itself. However for some reason provider also decides it's a good idea to store state there. But that bucket doesn't even exist in B at this point yet. Therefore, terraform errors out with a 403.

What are my options here? I've read through terraform docs and there is not a hint from what I've seen on how to force a specific provider configuration onto my backend.

New or Affected Resource(s)

• bleeds from google provider into gcs backend in terraform block

Potential Terraform Configuration

Given above

# All of my state goes into a bucket in A
terraform {
  backend "gcs" {
    bucket  = "I-want-this-state-in-A"
    prefix  = "terraform/state"
    # Once upon a time these options were available, but were removed at some point
    project = projectA
    region = regionA
  }
}

# All of my resources get spun up in B
provider "google" {
  region  = myregionB
  project = myprojectB
}

References

• #0000

[upodroid]

Hi

We do this at work alot where the state bucket is in a separate project to the provider project but the 403 happens because you don't have the correct roles on the state bucket.

Terraform executes in Go the following:

gsutil cp gs://somebucket/prefix/default.tfstate .
some gsutil rm commands too

Remember 403 is an issue with permissions where as 404 refers to not found. You will need to have Object Admin on the state bucket.

https://cloud.google.com/storage/docs/json_api/v1/status-codes

[danawillow]

I think @upodroid is right here. In addition, the gcs backend is actually part of the https://github.com/hashicorp/terraform repo, not the provider one. I'm going to go ahead and close this out- if the comments around checking permissions don't work, I'd recommend trying to file an issue there.

[Reifier]

No, the 403 is not related to the bucket. Project B is not spun up yet. When I spin it up without specifying any terraform backend everything is ok state is stored locally, project B gets up with setting specified in provider. However, as soon as I add a gcs backend:

Error: Error reading Project "Project-B": googleapi: Error 403: The caller does not have permission, forbidden

  on gcp.tf line 30, in data "google_project" "B":
  30: data "google_project" "B" {

So the project itself specified in provider which does not exist yet, not an existing bucket send me 403.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!