google_datafusion_instance as no zone argument in DEVELOPER mode

[Guimove]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Affected Resource(s)

• google_datafusion_instance

Terraform Configuration Files

resource "google_data_fusion_instance" "instance" {
  provider                      = google-beta
  name                          = var.name
  project                       = var.project
  description                   = var.description
  region                        = var.region
  type                          = "DEVELOPER"
  enable_stackdriver_logging    = true
  enable_stackdriver_monitoring = true
  labels                        = var.labels
  options                       = var.options
  private_instance              = var.network_config != null
  dataproc_service_account      = var.dataproc_service_account

  dynamic "network_config" {
    for_each = var.network_config == null ? [] : [var.network_config]
    content {
      network       = var.network_config.network
      ip_allocation = var.network_config.ip_allocation
    }
  }
}

Expected Behavior

We expect to deploy a data fusion instance in DEVELOPER edition within some minutes.

Actual Behavior

After 60 minutes, terraform apply fail with timeout. After checking in the GCP UI, I see the instance in "Creating..." status then disappear after another couple of minutes.

I suspect that the creation fail because we didn't specify any zone for this instance. This parameter is mandatory for a DEVELOPER edition (but inexistent for the others and for terraform).

This was already reported here : #8508 (comment)

Steps to Reproduce

1. terraform apply

Important Factoids

GCP Ref : https://cloud.google.com/sdk/gcloud/reference/beta/data-fusion/instances/create

[venkykuberan]

@Guimove I am able to get the resource created successfully, It took about 15 mins though ..
I used the following config and attached the API response as well. It seems like it took the default zone for the region.

Terraform version

Terraform v0.13.4
+ provider registry.terraform.io/hashicorp/google v3.64.0
+ provider registry.terraform.io/hashicorp/google-beta v3.64.0
+ provider registry.terraform.io/hashicorp/random v3.1.0

resource "google_data_fusion_instance" "extended_instance" {
  provider = google-beta
  name = "my-instance"
  description = "My Data Fusion instance"
  region = "us-central1"
  type = "DEVELOPER"
  enable_stackdriver_logging = true
  enable_stackdriver_monitoring = true
  labels = {
    example_key = "example_value"
  }
  private_instance = true
  network_config {
    network = "default"
    ip_allocation = "10.89.48.0/22"
  }
  version = "6.3.0"
  dataproc_service_account = data.google_app_engine_default_service_account.default.email
}

data "google_app_engine_default_service_account" "default" {
  provider = google-beta
}

API response

  {
  "name": "projects/test-project1-xxxxx/locations/us-central1/instances/my-instance",
  "description": "My Data Fusion instance",
  "type": "DEVELOPER",
  "enableStackdriverLogging": true,
  "enableStackdriverMonitoring": true,
  "privateInstance": true,
  "networkConfig": {
    "network": "default",
    "ipAllocation": "10.89.48.0/22"
  },
  "labels": {
    "example_key": "example_value"
  },
  "createTime": "2021-05-05T17:45:21.000376781Z",
  "updateTime": "2021-05-05T17:58:46.443664698Z",
  "state": "RUNNING",
  "serviceEndpoint": "https://my-instance-test-project1-xxxxx-dot-usc1.datafusion.googleusercontent.com",
  "zone": "us-central1-b",
  "version": "6.3.0",
  "serviceAccount": "cloud-datafusion-management-sa@k25d3e35dcc7e0b04p-tp.iam.gserviceaccount.com",
  "availableVersion": [
    {
      "versionNumber": "6.3.1"
    },
    {
      "versionNumber": "6.4.0"
    }
  ],
  "apiEndpoint": "https://my-instance-test-project1-xxxx-dot-usc1.datafusion.googleusercontent.com/api",
  "gcsBucket": "gs://df-xxxxx-m5qehtnnzii6xawqaizbbqaaaa",
  "p4ServiceAccount": "service-xxxx@gcp-sa-datafusion.iam.gserviceaccount.com",
  "tenantProjectId": "k25d3e35dcc7e0b04p-tp",
  "dataprocServiceAccount": "test-project1-xxx@appspot.gserviceaccount.com"
}

[Guimove]

mmh OK seems it also worked on my side today with my original code... maybe a temporary issue or something like.

But imo I think having the option to choose the zone could be great, what do you think ?

Like that you can stick some resources on the same zone or on the opposite, be sure that two instances are not on the same zone, for example.

[venkykuberan]

Agreed, will mark this as an enhancement request since it's not a bug. Team will triage it and take it from there.

[Guimove]

Thanks you, is it something also build by the magic modules ? if yes I've another PR ongoing so I can also take this point

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.