Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

google_storage_transfer_job requires schedule_end_date #3472

Closed
andycui66 opened this issue Apr 22, 2019 · 4 comments · Fixed by GoogleCloudPlatform/magic-modules#2044
Closed

google_storage_transfer_job requires schedule_end_date #3472

andycui66 opened this issue Apr 22, 2019 · 4 comments · Fixed by GoogleCloudPlatform/magic-modules#2044
Assignees
@rileykarson
Labels
bug forward/review In review; remove label to forward service/storage

Comments

@andycui66
Copy link
Contributor

andycui66 commented Apr 22, 2019 •
edited
Loading

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
  • If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.11.11

  • provider.google v2.5.0

Affected Resource(s)

  • resource "google_storage_transfer_job"

Terraform Configuration Files

variable "gcp_project_id" {}

variable "gcp_region" {}

variable "backup_bucket" {}

variable "backup_aws_access_key_id" {}

variable "backup_aws_secret_access_key" {}

provider "google" {
  version = "= 2.5.0"
  project = "${var.gcp_project_id}"
  region  = "${var.gcp_region}"
}

resource "google_storage_bucket" "backup-bucket" {
  name = "test-bucket"
}

resource "google_storage_transfer_job" "testing-backup-transfer-job" {
  description = "testing-backup-transfer-job"
  project     = "${var.gcp_project_id}"

  transfer_spec {
    aws_s3_data_source {
      bucket_name = "${var.backup_bucket}"

      aws_access_key {
        access_key_id     = "${var.backup_aws_access_key_id}"
        secret_access_key = "${var.backup_aws_secret_access_key}"
      }
    }

    gcs_data_sink {
      bucket_name = "${google_storage_bucket.backup-bucket.name}"
    }

    transfer_options {
      delete_objects_unique_in_sink = true
    }
  }

  schedule {
    schedule_start_date {
      year  = 2019
      month = 4
      day   = 22
    }
  }
}

Debug Output

2019-04-22T15:22:22.873-0400 [DEBUG] plugin.terraform-provider-google_v2.5.0_x4: 2019/04/22 15:22:22 [ERR] plugin: plugin server: accept unix /var/folders/n2/f8xxj94s1570n2r6m8bj0_nr0000gn/T/plugin100669011: use of closed network connection
* google_storage_transfer_job.testing-backup-transfer-job: googleapi: Error 400: Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive., badRequest

Expected Behavior

schedule_end_date is optional. Should be able to create a transfer job without it.

Actual Behavior

  • google_storage_transfer_job.testing-backup-transfer-job: googleapi: Error 400: Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive., badRequest

Steps to Reproduce

  1. terraform apply

Important Factoids

References

  • #0000
@ghost ghost added the bug label Apr 22, 2019
@akramhussein
Copy link
Contributor

Hi @andycui66 - I wrote the PR for this resource and have also found this issue recently.

When I initially wrote it schedule_end_date was and still is optional - code is still the same - so something has changed at the API layer from GCP.

@rileykarson if we can get a confirmation if the API has changed - I can submit a PR for this.

The API docs say schedule_start_date is required, but not schedule_end_date

@rileykarson
Copy link
Collaborator

Are you able to attach debug logs? I can't find anything indicating that you'd receive this error when the value was unset, only when it was invalid. Maybe Terraform is sending a different request than we expect?

Do you know when the behaviour changed? Based on the included provider version, could it have been in 2.5.0? That was the first provider version with the 0.12 SDK, so maybe 0-values are being handled differently.

@akramhussein
Copy link
Contributor

akramhussein commented Jul 3, 2019 •
edited
Loading

  • Worked for me on 20181126 but did not work on 20190528. Did not make any changes between then
  • Worked with 2.0.0
  • Now on 2.10.0 - does not work.

Here is the TF log snippet when it tries to create the resource with TF_LOG=DEBUG (I've scrubbed private data):

module.s3_backup_buckets.google_storage_transfer_job.s3_bucket_backup[12]: Creating...
2019/07/04 00:33:32 [DEBUG] module.s3_backup_buckets.google_storage_transfer_job.s3_bucket_backup[12]: applying the planned Create change
2019/07/04 00:33:32 [TRACE] GRPCProvider: ApplyResourceChange
2019-07-04T00:33:32.552+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: 2019/07/04 00:33:32 [DEBUG] Waiting for state to become: [success]
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: 2019/07/04 00:33:32 [DEBUG] Google API Request Details:
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: ---[ REQUEST ]---------------------------------------
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: POST /v1/transferJobs?alt=json&prettyPrint=false HTTP/1.1
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Host: storagetransfer.googleapis.com
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: User-Agent: google-api-go-client/0.5 Terraform/0.12.2 (+https://www.terraform.io) terraform-provider-google/2.10.0
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Content-Length: 580
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Content-Type: application/json
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Accept-Encoding: gzip
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  "description": "my-bucket-name-backup",
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  "projectId": "my-gcp-project",
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  "schedule": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "scheduleEndDate": {},
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "scheduleStartDate": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "day": 8,
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "month": 10,
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "year": 2018
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   },
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "startTimeOfDay": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "hours": 8
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   }
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  },
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  "status": "ENABLED",
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  "transferSpec": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "awsS3DataSource": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "awsAccessKey": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     "accessKeyId": "ACCESS_KEY_ID",
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     "secretAccessKey": "SECRET_ACCESS_KEY"
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    },
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "bucketName": "my-bucket-name"
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   },
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "gcsDataSink": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "bucketName": "my-bucket-name"
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   },
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "transferOptions": {
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:    "deleteObjectsUniqueInSink": true
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   }
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:  }
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: }
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:
2019-07-04T00:33:32.553+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: -----------------------------------------------------
2019-07-04T00:33:33.416+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: 2019/07/04 00:33:33 [DEBUG] Google API Response Details:
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: ---[ RESPONSE ]--------------------------------------
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: HTTP/2.0 400 Bad Request
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Alt-Svc: quic=":443"; ma=2592000; v="46,43,39"
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Cache-Control: private
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Content-Type: application/json; charset=UTF-8
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Date: Wed, 03 Jul 2019 23:33:33 GMT
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Server: ESF
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Vary: Origin
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Vary: X-Origin
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: Vary: Referer
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: X-Content-Type-Options: nosniff
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: X-Frame-Options: SAMEORIGIN
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: X-Xss-Protection: 0
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: {
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   "error": {
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     "code": 400,
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     "message": "Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive.",
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     "errors": [
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:       {
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:         "message": "Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive.",
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:         "domain": "global",
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:         "reason": "badRequest"
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:       }
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     ],
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:     "status": "INVALID_ARGUMENT"
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:   }
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: }
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4:
2019-07-04T00:33:33.417+0100 [DEBUG] plugin.terraform-provider-google_v2.10.0_x4: -----------------------------------------------------
2019/07/04 00:33:33 [DEBUG] module.s3_backup_buckets.google_storage_transfer_job.s3_bucket_backup[12]: apply errored, but we're indicating that via the Error pointer rather than returning it: googleapi: Error 400: Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive., badRequest
2019/07/04 00:33:33 [TRACE] module.s3_backup_buckets: eval: *terraform.EvalMaybeTainted
2019/07/04 00:33:33 [ERROR] module.s3_backup_buckets: eval: *terraform.EvalApplyPost, err: googleapi: Error 400: Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive., badRequest
2019/07/04 00:33:33 [ERROR] module.s3_backup_buckets: eval: *terraform.EvalSequence, err: googleapi: Error 400: Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive., badRequest

Error: googleapi: Error 400: Schedule end date: Invalid year specified. Year must be between 1 and 9999 inclusive., badRequest

  on ../../modules/storage/buckets/s3_backup/main.tf line 85, in resource "google_storage_transfer_job" "s3_bucket_backup":
  85: resource "google_storage_transfer_job" "s3_bucket_backup" {

Seems "scheduleEndDate": {}, is included when it shouldn't be.

@rileykarson rileykarson self-assigned this Jul 11, 2019
@rileykarson rileykarson mentioned this issue Jul 11, 2019
Merged
@ghost
Copy link

ghost commented Aug 15, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

@ghost ghost locked and limited conversation to collaborators Aug 15, 2019
@github-actions github-actions bot added service/storage forward/review In review; remove label to forward labels Jan 15, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@rileykarson rileykarson

Labels
bug forward/review In review; remove label to forward service/storage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix unnecessary nesting in google_storage_transfer_job rileykarson/magic-modules
3 participants
@akramhussein @rileykarson @andycui66