Composer - Http error status code: 400

[abdul-git]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.12.20

• provider.google v3.19.0
• provider.google-beta v3.19.0

Your version of Terraform is out of date! The latest version
is 0.12.24. You can update by downloading from https://www.terraform.io/downloads.html

Affected Resource(s)

google_composer_environment

Terraform Configuration Files

Composer - TF 
resource "google_composer_environment" "composer01" {
  name   = var.cluster_composer_name
  region = var.region
  config {
    node_count = var.node_count

    node_config {
      zone         = "${var.region}-a"
      machine_type = "n1-standard-1"

      network    = google_compute_network.composer_net.id
      subnetwork = google_compute_subnetwork.composer_net_subnet.id

      service_account = google_service_account.test.name
    }
  }

  depends_on = [google_project_iam_member.composer-worker]
}

Network - TF
resource "google_compute_network" "composer_net" {
  name                    = "composer-test-network"
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "composer_net_subnet" {
  name          = "composer-test-subnetwork"
  ip_cidr_range = "10.2.0.0/16"
  region        = var.region
  network       = google_compute_network.composer_net.id
}

IAM - TF 
resource "google_service_account" "test" {
  account_id   = "composer-env-account"
  display_name = "Test Service Account for Composer Environment"
}

resource "google_project_iam_member" "composer-worker" {
  role   = "roles/composer.worker"
  member = "serviceAccount:${google_service_account.test.email}"

Debug Output

Error: Error waiting to create Environment: Error waiting for Creating Environment: Error code 3, message: Http error status code: 400
Http error message: BAD REQUEST
Additional errors:
{"ResourceType":"pw-securectl-host-prj/us-central1-my-composer-001-46ff1078-addons-gke-typer:container.projects.zones.clusters.nodePools.setManagement","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Cluster is currently being created, deleted, updated or repaired and cannot be updated.","status":"FAILED_PRECONDITION","details":[],"statusMessage":"Bad Request","requestPath":"https://container.googleapis.com/v1/projects/pw-securectl-host-prj/zones/us-central1-a/clusters/us-central1-my-composer-001-46ff1078-gke/nodePools/default-pool/setManagement","httpMethod":"POST"}}

on composer.tf line 1, in resource "google_composer_environment" "composer01":
1: resource "google_composer_environment" "composer01" {

https://gist.github.com/abdul-git/909d252fbd48c5e2b744b6831275707a

https://gist.github.com/abdul-git/19e3a82f8f1d4bfc990dbd2b9f8b4cac

Panic Output

Expected Behavior

Error 400 after 55 mins of build time and terminates - unable to create using the following:

https://www.terraform.io/docs/providers/google/r/composer_environment.html

Actual Behavior

Steps to Reproduce

1. terraform apply

Important Factoids

References

• google_composer_environment: bucket resource not found? #3814

[edwardmedia]

@abdul-git can you share your full debug log so I can take a closer look? Without additional info available, from what the error says Cluster is currently being created, deleted, updated or repaired and cannot be updated, did you apply the code already?

[abdul-git]

composer.log.editor.failed.gz
composer.log.failed.gz

[edwardmedia]

@abdul-git can you post logs at https://gist.github.com/? Thanks

[abdul-git]

@edwardmedia gist added to the debug section.

[abdul-git]

Hello! any update?

[edwardmedia]

@abdul-git thanks for posting the logs. The error says Cluster is currently being created, deleted, updated or repaired and cannot be updated, did you apply the code already? I can't reach that error. Please provide steps that I can follow to repro it.

[abdul-git]

@edwardmedia it just terminates while creating. The steps are the following:

- terraform init

- terraform plan

- terraform apply

google_composer_environment.composer-builder: Still creating... [53m10s elapsed]
google_composer_environment.composer-builder: Still creating... [53m20s elapsed]
google_composer_environment.composer-builder: Still creating... [53m30s elapsed]
google_composer_environment.composer-builder: Still creating... [53m40s elapsed]
google_composer_environment.composer-builder: Still creating... [53m50s elapsed]
google_composer_environment.composer-builder: Still creating... [54m0s elapsed]
google_composer_environment.composer-builder: Still creating... [54m10s elapsed]

Error: Error waiting to create Environment: Error waiting for Creating Environment: Error code 3, message: Http error status code: 400
Http error message: BAD REQUEST
Additional errors:
{"ResourceType":"gcp-sctl-datahub-002/us-east4-composer-0002-b88ac5fc-addons-gke-typer:container.projects.zones.clusters.nodePools.setManagement","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Cluster is currently being created, deleted, updated or repaired and cannot be updated.","status":"FAILED_PRECONDITION","details":[],"statusMessage":"Bad Request","requestPath":"https://container.googleapis.com/v1/projects/gcp-sctl-datahub-002/zones/us-east4-b/clusters/us-east4-composer-0002-b88ac5fc-gke/nodePools/default-pool/setManagement","httpMethod":"POST"}}

on composer.tf line 15, in resource "google_composer_environment" "composer-builder":
15: resource "google_composer_environment" "composer-builder" {

[abdul-git]

Also, I can confirm same issue in customer environment - during the build it spins up GKE, but in final phase before the error cluster gets dropped.

[edwardmedia]

@abdul-git I successfully built a composer using your code without hitting an issue. Back to your log, I noticed below request. What is the network vpc-composer-002? I guess I don't have a full picture of your code here. Did you apply above code before?

2020-05-04T10:50:02.184-0500 [DEBUG] plugin.terraform-provider-google_v3.19.0_x5: ---[ REQUEST ]---------------------------------------
2020-05-04T10:50:02.184-0500 [DEBUG] plugin.terraform-provider-google_v3.19.0_x5: DELETE /compute/v1/projects/pw-securectl-host-prj/global/networks/vpc-composer-002?alt=json HTTP/1.1

[abdul-git]

@edwardmedia I have figured out what the issue is! it seems org level constraint was causing the issue with external IP, after disabling org level constraint I was able to create it successfully.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!