Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add event_stream block to google_storage_transfer_job schema #8894

Conversation

BBBmau
Copy link
Collaborator

@BBBmau BBBmau commented Sep 8, 2023

This PR adds the event_stream field to the google_storage_transfer_job resource. This can be found in the docs here

It's also referenced as part of a feature request here: hashicorp/terraform-provider-google#13293

Release Note Template for Downstream PRs (will be copied)

storagetransfer: added `event_stream` field to `google_storage_transfer_job` resource

@modular-magician
Copy link
Collaborator

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will require approval to run.

@SarahFrench, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@modular-magician modular-magician added awaiting-approval Pull requests that need reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels Sep 8, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 29 insertions(+))
Terraform Beta: Diff ( 1 file changed, 29 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_storage_transfer_job (17 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_storage_transfer_job" "primary" {
  event_stream {
    event_stream_expiration_time = # value needed
    event_stream_start_time      = # value needed
    name                         = # value needed
  }
}

@SarahFrench
Copy link
Contributor

@BBBmau I kicked off the automated tests, though I'm conscious that this is still a draft so won't review until you ask/mark as ready!

Also, take a look at the guidance about writing release notes - if you were a community contributor I would edit your release note for you, but I'd like you to have a go getting the release note to match our style guide as practice.

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3018
Passed tests 2714
Skipped tests: 297
Affected tests: 7

Action taken

Found 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccStorageTransferJob_omitScheduleEndDate|TestAccStorageTransferJob_basic|TestAccStorageTransferJob_posixSink|TestAccStorageTransferJob_objectConditions|TestAccStorageTransferJob_notificationConfig|TestAccStorageTransferJob_posixSource|TestAccStorageTransferJob_transferOptions

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccStorageTransferJob_omitScheduleEndDate[Error message] [Debug log]
TestAccStorageTransferJob_basic[Error message] [Debug log]
TestAccStorageTransferJob_posixSink[Error message] [Debug log]
TestAccStorageTransferJob_objectConditions[Error message] [Debug log]
TestAccStorageTransferJob_notificationConfig[Error message] [Debug log]
TestAccStorageTransferJob_posixSource[Error message] [Debug log]
TestAccStorageTransferJob_transferOptions[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@modular-magician modular-magician added awaiting-approval Pull requests that need reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels Sep 12, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 164 insertions(+))
Terraform Beta: Diff ( 3 files changed, 164 insertions(+))

Copy link
Contributor

@SarahFrench SarahFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick fly-by review while we wait for the tests to run, I'll give more feedback once they finish!

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3037
Passed tests 2732
Skipped tests: 297
Affected tests: 8

Action taken

Found 8 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccBigtableAppProfile_bigtableAppProfileSingleclusterExample|TestAccGKEHub2Scope_gkehubScopeBasicExample|TestAccGKEHub2ScopeIamBindingGenerated|TestAccGKEHub2ScopeIamMemberGenerated|TestAccGKEHub2ScopeIamPolicyGenerated|TestAccStorageTransferJob_basic

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileSingleclusterExample[Debug log]
TestAccGKEHub2Scope_gkehubScopeBasicExample[Debug log]
TestAccGKEHub2ScopeIamBindingGenerated[Debug log]
TestAccGKEHub2ScopeIamMemberGenerated[Debug log]
TestAccGKEHub2ScopeIamPolicyGenerated[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccStorageTransferJob_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@modular-magician modular-magician added the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Sep 13, 2023
@BBBmau BBBmau marked this pull request as ready for review September 13, 2023 16:40
@SarahFrench
Copy link
Contributor

@BBBmau The test failure provided a useful error message:

Error: googleapi: Error 400: Transfer job can either have a schedule or event stream and not both., badRequest

This happened after the API call to create the resource:

---[ REQUEST ]---------------------------------------
POST /v1/transferJobs?alt=json&prettyPrint=false HTTP/1.1
Host: storagetransfer.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.2.5 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google-beta/acc
Content-Length: 535
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.19.12 gdcl/0.138.0
Accept-Encoding: gzip

{
 "description": "2s4ymgegui",
 "eventStream": {
  "eventStreamExpirationTime": "2023-10-02T15:01:23Z",
  "eventStreamStartTime": "2014-10-02T15:01:23Z",
  "name": "event_stream"
 },
 "projectId": "ci-test-project-188019",
 "schedule": {
  "repeatInterval": "604800s",
  "scheduleEndDate": {
   "day": 1,
   "month": 10,
   "year": 2019
  },
  "scheduleStartDate": {
   "day": 1,
   "month": 10,
   "year": 2018
  },
  "startTimeOfDay": {
   "minutes": 30
  }
 },
 "status": "ENABLED",
 "transferSpec": {
  "gcsDataSink": {
   "bucketName": "ndi9s48fa1",
   "path": "foo/bar/"
  },
  "gcsDataSource": {
   "bucketName": "lf6zrp28wl",
   "path": "foo/bar/"
  }
 }
}

@BBBmau
Copy link
Collaborator Author

BBBmau commented Sep 13, 2023

@BBBmau The test failure provided a useful error message:

Error: googleapi: Error 400: Transfer job can either have a schedule or event stream and not both., badRequest

Grateful for the clear Error message 🙏

We'll have to update the docs also since it has no reference of not being able to use both. The bottom is what is shown in the docs for both fields. @SarahFrench

schedule	
object ([Schedule](https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs#Schedule))

Specifies schedule for the transfer job. This is an optional field. When the field is not set, the job never executes a transfer, unless you invoke transferJobs.run or update the job to have a non-empty schedule.

eventStream	
object ([EventStream](https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs#EventStream))

Specifies the event stream for the transfer job for event-driven transfers. When EventStream is specified, the Schedule fields are ignored.

@BBBmau
Copy link
Collaborator Author

BBBmau commented Sep 13, 2023

@SarahFrench
Copy link
Contributor

I also noticed that schedule is a Required field so we'll need to change it to Optional since it's either schedule or event_stream but not both or neither. At least one must be set.

Good catch! Looks like it's marked that way in the docs but not the code (phew) so we're not expanding the scope of the PR too much

@modular-magician modular-magician added awaiting-approval Pull requests that need reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels Sep 22, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 233 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 3 files changed, 233 insertions(+), 5 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3080
Passed tests 2780
Skipped tests: 299
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccStorageTransferJob_eventStream

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccStorageTransferJob_eventStream[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

Copy link
Contributor

@SarahFrench SarahFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats on getting the acceptance test passing! Now we need to ensure the test will run robustly. See individual comments for details 😁

Also, take a look at the guidance about writing release notes - if you were a community contributor I would edit your release note for you, but I'd like you to have a go getting the release note to match our style guide as practice.

…torage_transfer_job_test.go

Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
@modular-magician modular-magician added the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Sep 22, 2023
@modular-magician modular-magician removed the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Sep 25, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 242 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 3 files changed, 237 insertions(+), 5 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3081
Passed tests 2781
Skipped tests: 299
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccStorageTransferJob_eventStream

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccStorageTransferJob_eventStream[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

@modular-magician modular-magician added awaiting-approval Pull requests that need reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels Sep 26, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 235 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 3 files changed, 235 insertions(+), 5 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3092
Passed tests 2792
Skipped tests: 299
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccDataprocClusterIamPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataprocClusterIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

@SarahFrench
Copy link
Contributor

Thanks for making that final change - I'm happy to approve and merge now.

⚠️ Please read @BBBmau : I've going to update the release note in your PR description as it doesn't match our style guide. Please take a look at our guidance about writing release notes when making the release note for other Google provider PRs.

On that page it says this is how release notes for new fields should be added:

```release-note:enhancement
PRODUCT: added `FIELD_1`, `FIELD_2`, and `FIELD_N` fields to `RESOURCE_NAME` resource
```

So instead of:

mmv1/storagetransfer/google_storage_transfer_job.go: add event_stream field

I've changed this PR's release note to:

storagetransfer: added event_stream field to google_storage_transfer_job resource

Copy link
Contributor

@SarahFrench SarahFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants