Support on-demand schedule on google_bigquery_data_transfer_config

[avegao]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

From the BigQuery Transfer web console I can create schedules on demand and with Terraform this is not supported with the schedule property, which only supports schedules and not some value like on_demand.

Seeing how the web console works, I have found this structure that has the scheduleOptions property that is not supported by Terraform:

{
  "name": "projects/PROJECT/locations/LOCATION/transferConfigs/ID",
  "destinationDatasetId": "dataset",
  "displayName": "new_transfer",
  "updateTime": "2020-10-01T11:41:00.328726Z",
  "dataSourceId": "amazon_s3",
  "params": {
    "destination_table_name_template": "table",
    "access_key_id": "KEY",
    "data_path": "s3://BUCKET",
    "max_bad_records": "0",
    "file_format": "PARQUET",
    "secret_access_key": "SECRET"
  },
  "state": "FAILED",
  "userId": "ID",
  "datasetRegion": "europe",
  "emailPreferences": {},
  "createTime": "2020-10-01T11:19:32.691449187Z",
  "scheduleOptions": {
    "disableAutoScheduling": true
  },
  "manualRunScheduleType": "REFRESH_TRANSFER"
}

New or Affected Resource(s)

• google_bigquery_data_transfer_config

Potential Terraform Configuration

I have 2 proposal:

resource "google_bigquery_data_transfer_config" "transfer" {
  display_name           = "new_transfer"
  location               = "EU"
  data_source_id         = "amazon_s3"
  destination_dataset_id = "dataset"

  params = {
    destination_table_name_template = "table"
    write_disposition               = "WRITE_APPEND"
    access_key_id                   = "KEY"
    data_path                       = "s3:/BUCKET"
    file_format                     = "PARQUET"
    max_bad_records                 = 0
    write_disposition               = null
  }

  schedule_options {
    disable_auto_scheduling = true
  }

  sensitive_params {
    secret_access_key = "SECRET"
  }
}

New option:

 schedule_options {
    disable_auto_scheduling = true
  }

resource "google_bigquery_data_transfer_config" "transfer" {
  display_name           = "new_transfer"
  location               = "EU"
  data_source_id         = "amazon_s3"
  destination_dataset_id = "dataset"
  schedule               = "on_demand"

  params = {
    destination_table_name_template = "table"
    write_disposition               = "WRITE_APPEND"
    access_key_id                   = "KEY"
    data_path                       = "s3:/BUCKET"
    file_format                     = "PARQUET"
    max_bad_records                 = 0
    write_disposition               = null
  }

  sensitive_params {
    secret_access_key = "SECRET"
  }
}

schedule = "on_demand"

[avegao]

This request is included in v3.46.0 in #7633

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!