Cannot move access from google_bigquery_dataset

[olib963]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Currently terraform 0.13.5 and google provider 3.44.0, however is still an issue when attempted with 0.14.4 and 3.51.1 respectively.

Affected Resource(s)

• google_bigquery_dataset

Terraform Configuration Files

# Before
resource "google_bigquery_dataset" "private" {
  dataset_id    = "test-dataset"
  friendly_name = "Test"
  description   = "Testing BQ Permissions"
  location      = "EU"

   access {
    role          = "OWNER"
    user_by_email = google_service_account.bqowner.email
  }

  access {
    role   = "READER"
    domain = "hashicorp.com"
  }
}

resource "google_service_account" "bqowner" {
  account_id = "bqowner"
}

# After
resource "google_bigquery_dataset" "private" {
  dataset_id    = "test-dataset"
  friendly_name = "Test"
  description   = "Testing BQ Permissions"
  location      = "EU"
}

#resource "google_bigquery_dataset_access" "reader" {
#  dataset_id    = google_bigquery_dataset.private.dataset_id
#  role          = "READER"
#  domain = "hashicorp.com"
#}

#resource "google_bigquery_dataset_access" "owner" {
#  dataset_id    = google_bigquery_dataset.private.dataset_id
#  role          = "OWNER"
#  user_by_email = google_service_account.bqowner.email
#}

resource "google_service_account" "bqowner" {
  account_id = "bqowner"
}

Expected Behavior

I should be able to move the access definition from the dataset itself to separate resources. The reason for doing this is that currently we have a lot of access blocks and a change in any of them results in an ugly plan that removes all access blocks, even the ones that don't change, then adds all the correct blocks. I thought that having each access block live on its own would make the changes in state much clearer.

Actual Behavior

Removing all access blocks from a dataset however results in an empty plan and you cannot set access = [] since that results in a terraform error asking if "you meant you define an "access" block". This means we are unable to transition to the google_bigquery_dataset_access resource.

Steps to Reproduce

1. terraform apply the "before" terraform file
2. terraform plan or terraform apply the "after" file to see that no changes are detected

Ideally there would then be a step 3: uncomment new resources and terraform apply again to get access back.

[slevenick]

Hey @olib963 does adding ignore_changes work for your use case? It should cause Terraform to ignore any changes to the access block, preventing the diff on the main dataset resource.

https://www.terraform.io/docs/configuration/meta-arguments/lifecycle.html#ignore_changes

I don't believe there is a better way short of destroying and recreating the dataset with access unspecified

[olib963]

I can't ignore changes since the google_bigquery_dataset_access currently fails to create because the access is already defined on the dataset itself. I'd like to not destroy and recreate since there's a lot of data in there. I'd have to come up with some process to stop writing new data and back it up then copy it back in.

Do we know why it is the case that removing all access blocks does not show a change in the desired state? For example if I change the resource to

resource "google_bigquery_dataset" "private" {
  dataset_id    = "test-dataset"
  friendly_name = "Test"
  description   = "Testing BQ Permissions"
  location      = "EU"

   access {
    role          = "OWNER"
    user_by_email = google_service_account.bqowner.email
  }
}

It completely understands it needs to remove the READER access, but removing both leads to an empty plan.

[slevenick]

Yeah, when you remove the access block entirely it interprets that as "I don't care about what access is" rather than "access should be empty". This is important because once you manage access with the google_bigquery_dataset_access resource, we don't want the bigquery dataset to revert any changes. Is this empty plan a problem? I'd suggest that you could remove the access manually and recreate using the google_bigquery_dataset_access resource. It's a bit of an annoying step, but could get you to the world where the dataset does not manage access, and the access resource does.

By moving to using the google_bigquery_dataset_access resource you are moving to a non-authoritative permissions model, meaning that there could be additional access/permissions on the dataset that will not be removed by Terraform. If you want to continue using an authoritative model you could look into using https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_dataset_iam
Note: this will not allow some advanced usages for datasets

[slevenick]

Also, I believe this feature request is for something that would help moving between these resources: #7486

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!