Add Vertex ML Metadata Store #4951

upodroid · 2021-07-05T18:08:23Z

Part of: hashicorp/terraform-provider-google#9298

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

`google_vertex_ai_metadata_store`

modular-magician · 2021-07-05T18:10:03Z

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@c2thorn, please review this PR or find an appropriate assignee.

upodroid · 2021-07-05T18:10:31Z

There is a bug in the API. It doesn't return encryption_spec fields.

The whole reason for using this resource is to support CMEK and the API doesn't even return those fields 😕

021/07/05 19:04:19 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
POST /v1beta1/projects/REDACTED/locations/us-central1/metadataStores?alt=json&metadataStoreId=tf-test-2weojyw8lj HTTP/1.1
Host: us-central1-aiplatform.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google-beta/acc
Content-Length: 169
Content-Type: application/json
Accept-Encoding: gzip

{
 "description": "Magic",
 "encryptionSpec": {
  "kmsKeyName": "projects/REDACTED/locations/us-central1/keyRings/tftest-shared-keyring-1/cryptoKeys/tftest-shared-key-1"
 }
}

-----------------------------------------------------
2021/07/05 19:04:20 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 05 Jul 2021 18:04:20 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "projects/550924169191/locations/us-central1/operations/616599798664921088",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.CreateMetadataStoreOperationMetadata",
    "genericMetadata": {
      "createTime": "2021-07-05T18:04:20.582962Z",
      "updateTime": "2021-07-05T18:04:20.582962Z"
    }
  }
}

-----------------------------------------------------
2021/07/05 19:04:20 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/07/05 19:04:20 [DEBUG] Retry Transport: Returning after 1 attempts
2021/07/05 19:04:20 [DEBUG] Waiting for state to become: [done: true]
2021/07/05 19:04:20 [DEBUG] Waiting for state to become: [success]
2021/07/05 19:04:20 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/07/05 19:04:20 [DEBUG] Retry Transport: request attempt 0
2021/07/05 19:04:20 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1beta1/projects/550924169191/locations/us-central1/operations/616599798664921088?alt=json HTTP/1.1
Host: us-central1-aiplatform.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google-beta/acc
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------
2021/07/05 19:04:21 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 05 Jul 2021 18:04:21 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "projects/550924169191/locations/us-central1/operations/616599798664921088",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.CreateMetadataStoreOperationMetadata",
    "genericMetadata": {
      "createTime": "2021-07-05T18:04:20.582962Z",
      "updateTime": "2021-07-05T18:04:20.582962Z"
    }
  }
}

-----------------------------------------------------
2021/07/05 19:04:21 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/07/05 19:04:21 [DEBUG] Retry Transport: Returning after 1 attempts
2021/07/05 19:04:21 [DEBUG] Got done: false while polling for operation projects/550924169191/locations/us-central1/operations/616599798664921088's status
2021/07/05 19:04:31 [DEBUG] Waiting for state to become: [success]
2021/07/05 19:04:31 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/07/05 19:04:31 [DEBUG] Retry Transport: request attempt 0
2021/07/05 19:04:31 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1beta1/projects/550924169191/locations/us-central1/operations/616599798664921088?alt=json HTTP/1.1
Host: us-central1-aiplatform.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google-beta/acc
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------
2021/07/05 19:04:31 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 05 Jul 2021 18:04:31 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "projects/550924169191/locations/us-central1/operations/616599798664921088",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.CreateMetadataStoreOperationMetadata",
    "genericMetadata": {
      "createTime": "2021-07-05T18:04:20.582962Z",
      "updateTime": "2021-07-05T18:04:21.980978Z"
    }
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.MetadataStore",
    "name": "projects/550924169191/locations/us-central1/metadataStores/tf-test-2weojyw8lj"
  }
}

-----------------------------------------------------
2021/07/05 19:04:31 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/07/05 19:04:31 [DEBUG] Retry Transport: Returning after 1 attempts
2021/07/05 19:04:31 [DEBUG] Got done: true while polling for operation projects/550924169191/locations/us-central1/operations/616599798664921088's status
2021/07/05 19:04:31 [DEBUG] Finished creating MetadataStore "tf-test-2weojyw8lj": map[string]interface {}{"metadata":map[string]interface {}{"@type":"type.googleapis.com/google.cloud.aiplatform.v1beta1.CreateMetadataStoreOperationMetadata", "genericMetadata":map[string]interface {}{"createTime":"2021-07-05T18:04:20.582962Z", "updateTime":"2021-07-05T18:04:20.582962Z"}}, "name":"projects/550924169191/locations/us-central1/operations/616599798664921088"}
2021/07/05 19:04:31 [DEBUG] Waiting for state to become: [success]
2021/07/05 19:04:31 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/07/05 19:04:31 [DEBUG] Retry Transport: request attempt 0
2021/07/05 19:04:31 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1beta1/projects/REDACTED/locations/us-central1/metadataStores/tf-test-2weojyw8lj?alt=json HTTP/1.1
Host: us-central1-aiplatform.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google-beta/acc
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------
2021/07/05 19:04:32 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 05 Jul 2021 18:04:32 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "projects/550924169191/locations/us-central1/metadataStores/tf-test-2weojyw8lj",
  "createTime": "2021-07-05T18:04:20.582962Z",
  "updateTime": "2021-07-05T18:04:22.015587Z",
  "state": {
    "diskUtilizationBytes": "851968"
  }
}

-----------------------------------------------------
2021/07/05 19:04:32 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/07/05 19:04:32 [DEBUG] Retry Transport: Returning after 1 attempts
2021/07/05 19:04:37 [INFO] Authenticating using DefaultClient...
2021/07/05 19:04:37 [INFO]   -- Scopes: [https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/cloud-identity https://www.googleapis.com/auth/ndev.clouddns.readwrite https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/userinfo.email]
2021/07/05 19:04:37 [INFO] Authenticating using DefaultClient...
2021/07/05 19:04:37 [INFO]   -- Scopes: [https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/cloud-identity https://www.googleapis.com/auth/ndev.clouddns.readwrite https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/userinfo.email]
2021/07/05 19:04:37 [DEBUG] Waiting for state to become: [success]
2021/07/05 19:04:37 [INFO] Terraform is using this identity: REDACTED
    provider_test.go:276: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:
        
        
        An execution plan has been generated and is shown below.
        Resource actions are indicated with the following symbols:
        -/+ destroy and then create replacement
        
        Terraform will perform the following actions:
        
          # google_vertex_ai_metadata_store.store must be replaced
        -/+ resource "google_vertex_ai_metadata_store" "store" {
              ~ create_time = "2021-07-05T18:04:20.582962Z" -> (known after apply)
              + description = "Magic"
              ~ id          = "tf-test-2weojyw8lj" -> (known after apply)
                name        = "tf-test-2weojyw8lj"
              ~ project     = "REDACTED" -> (known after apply)
              ~ state       = [
                  - {
                      - disk_utilization_bytes = "851968"
                    },
                ] -> (known after apply)
              ~ update_time = "2021-07-05T18:04:22.015587Z" -> (known after apply)
                # (1 unchanged attribute hidden)
        
              + encryption_spec {
                  + kms_key_name = "projects/REDACTED/locations/us-central1/keyRings/tftest-shared-keyring-1/cryptoKeys/tftest-shared-key-1"
                }
            }
        
        Plan: 1 to add, 0 to change, 1 to destroy.

upodroid · 2021-07-06T10:43:38Z

Filed b/192870434

c2thorn · 2021-07-08T16:16:01Z

Filed b/192870434

Thanks @upodroid. Following up on an internal tracker that points to yours.

modular-magician · 2021-08-10T16:50:22Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 133 insertions(+))
Terraform Beta: Diff ( 7 files changed, 599 insertions(+), 5 deletions(-))

upodroid · 2021-08-10T17:06:57Z

Google have said this is fixed but it is still active. I fixed the conflicts in the meanwhile.

b/193651954

jialuzh · 2021-11-17T18:13:01Z

@upodroid I am from Vertex ML Metadata team. b/193651954 should be fixed now, could you help retry it and let us know whether you still saw the issue?

c2thorn · 2021-11-17T21:39:50Z

/gcbrun

c2thorn · 2021-11-18T18:07:29Z

/gcbrun

modular-magician · 2021-11-18T19:50:35Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDataSourceComposerEnvironment_basic|TestAccDataSourceComputeAddress|TestAccDataSourceGoogleSQLCaCerts_basic|TestAccDataSourceSqlDatabaseInstance_basic|TestAccAssuredWorkloadsWorkload_FullHandWritten|TestAccCloudbuildWorkerPool_withNetwork|TestAccComposerEnvironment_update|TestAccComposerEnvironment_withSoftwareConfig|TestAccComputeInstanceFromMachineImage_basic|TestAccComputeInstanceFromMachineImage_overrideMetadataDotStartupScript|TestAccComputeInstanceFromTemplate_012_removableFields|TestAccComputeRegionNetworkEndpointGroup_regionNetworkEndpointGroupAppengineExample|TestAccContainerNodePool_withInvalidUpgradeSettings|TestAccContainerNodePool_ephemeralStorageConfig|TestAccNetworkServicesEdgeCacheOrigin_networkServicesEdgeCacheOriginAdvancedExample|TestAccNetworkServicesEdgeCacheService_networkServicesEdgeCacheServiceAdvancedExample|TestAccNetworkServicesEdgeCacheService_networkServicesEdgeCacheServiceBasicExample|TestAccNetworkServicesEdgeCacheService_updateAndImport|TestAccPrivatecaCaPool_privatecaCapoolAllFieldsExample|TestAccPrivatecaCaPool_privatecaCapoolUpdate|TestAccPrivatecaCaPool_privatecaCapoolEmptyBaseline|TestAccPrivatecaCertificate_privatecaCertificateNoAuthorityExample|TestAccPubsubSchema_pubsubSchemaBasicExample|TestAccStorageBucket_update|TestAccTPUNode_tpuNodeBUpdateTensorFlowVersion|TestAccVertexAIMetadataStore_vertexAiMetadataStoreExample You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=220964

modular-magician · 2021-11-19T16:30:24Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 133 insertions(+))
Terraform Beta: Diff ( 5 files changed, 598 insertions(+), 2 deletions(-))

Co-authored-by: upodroid <cy@borg.dev>

c2thorn · 2021-11-30T20:24:51Z

/gcbrun

modular-magician · 2021-11-30T20:32:54Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 133 insertions(+))
Terraform Beta: Diff ( 5 files changed, 598 insertions(+), 2 deletions(-))

modular-magician · 2021-11-30T20:35:47Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 133 insertions(+))
Terraform Beta: Diff ( 5 files changed, 598 insertions(+), 2 deletions(-))

c2thorn · 2021-11-30T22:24:13Z

/gcbrun

modular-magician · 2021-11-30T22:35:23Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 133 insertions(+))
Terraform Beta: Diff ( 5 files changed, 595 insertions(+), 2 deletions(-))

modular-magician · 2021-11-30T22:35:45Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 133 insertions(+))
Terraform Beta: Diff ( 5 files changed, 595 insertions(+), 2 deletions(-))

c2thorn · 2021-11-30T22:41:15Z

Tested locally - merging ahead of VCR tests to ensure the change is included in the next release.

upodroid · 2021-11-30T23:43:46Z

Thank you for getting this over the line.

Co-authored-by: upodroid <cy@borg.dev> Co-authored-by: Cameron Thornton <camthornton@google.com>

google-cla bot added the cla: yes label Jul 5, 2021

modular-magician requested a review from c2thorn July 5, 2021 18:10

This comment has been minimized.

Sign in to view

c2thorn force-pushed the vertex-ai-metadata-store branch from f4aca1d to 70c9c8e Compare November 19, 2021 16:19

add metadata store

9abdff0

Co-authored-by: upodroid <cy@borg.dev>

c2thorn force-pushed the vertex-ai-metadata-store branch from 70c9c8e to 9abdff0 Compare November 30, 2021 20:21

fix test imports

c9fef4d

c2thorn approved these changes Nov 30, 2021

View reviewed changes

c2thorn merged commit 042008a into GoogleCloudPlatform:master Nov 30, 2021

c2thorn deleted the vertex-ai-metadata-store branch November 30, 2021 22:41

This was referenced Nov 30, 2021

Add Vertex ML Metadata Store hashicorp/terraform-provider-google-beta#3885

Merged

Add Vertex ML Metadata Store hashicorp/terraform-provider-google#10657

Merged

c2thorn restored the vertex-ai-metadata-store branch December 8, 2021 20:55

upodroid mentioned this pull request Apr 12, 2022

Add support for Cloud Deploy resources #5710

Merged

5 tasks

betsy-lichtenberg pushed a commit to betsy-lichtenberg/magic-modules that referenced this pull request Apr 25, 2022

Add Vertex ML Metadata Store (GoogleCloudPlatform#4951)

fdb3c12

Co-authored-by: upodroid <cy@borg.dev> Co-authored-by: Cameron Thornton <camthornton@google.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Vertex ML Metadata Store #4951

Add Vertex ML Metadata Store #4951

upodroid commented Jul 5, 2021 •

edited

Loading

modular-magician commented Jul 5, 2021

upodroid commented Jul 5, 2021

upodroid commented Jul 6, 2021

c2thorn commented Jul 8, 2021

modular-magician commented Aug 10, 2021

upodroid commented Aug 10, 2021 •

edited

Loading

jialuzh commented Nov 17, 2021

c2thorn commented Nov 17, 2021

This comment has been minimized.

c2thorn commented Nov 18, 2021

modular-magician commented Nov 18, 2021

modular-magician commented Nov 19, 2021

c2thorn commented Nov 30, 2021

modular-magician commented Nov 30, 2021

modular-magician commented Nov 30, 2021

c2thorn commented Nov 30, 2021

modular-magician commented Nov 30, 2021

modular-magician commented Nov 30, 2021

c2thorn commented Nov 30, 2021

upodroid commented Nov 30, 2021

Add Vertex ML Metadata Store #4951

Add Vertex ML Metadata Store #4951

Conversation

upodroid commented Jul 5, 2021 • edited Loading

modular-magician commented Jul 5, 2021

upodroid commented Jul 5, 2021

upodroid commented Jul 6, 2021

c2thorn commented Jul 8, 2021

modular-magician commented Aug 10, 2021

Diff report:

upodroid commented Aug 10, 2021 • edited Loading

jialuzh commented Nov 17, 2021

c2thorn commented Nov 17, 2021

This comment has been minimized.

c2thorn commented Nov 18, 2021

modular-magician commented Nov 18, 2021

modular-magician commented Nov 19, 2021

Diff report:

c2thorn commented Nov 30, 2021

modular-magician commented Nov 30, 2021

Diff report:

modular-magician commented Nov 30, 2021

Diff report:

c2thorn commented Nov 30, 2021

modular-magician commented Nov 30, 2021

Diff report:

modular-magician commented Nov 30, 2021

Diff report:

c2thorn commented Nov 30, 2021

upodroid commented Nov 30, 2021

upodroid commented Jul 5, 2021 •

edited

Loading

upodroid commented Aug 10, 2021 •

edited

Loading