Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added data source google kms secret asymmetric #4609

Merged
merged 7 commits into from
Mar 22, 2021

Conversation

melinath
Copy link
Member

@melinath melinath commented Mar 19, 2021

Upstreams hashicorp/terraform-provider-google-beta#3052. Note that although the original PR was against tpg-beta, it seems to be a GA API.

Related to hashicorp/terraform-provider-google#6462.

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

`google_kms_secret_asymmetric`

@google-cla

This comment has been minimized.

@google-cla google-cla bot added the cla: no label Mar 19, 2021
@modular-magician

This comment has been minimized.

@modular-magician

This comment has been minimized.

@melinath
Copy link
Member Author

/gcbrun - agent failed to spin up

@melinath
Copy link
Member Author

CLA not required - all in third_party

@melinath melinath requested review from a team and c2thorn and removed request for a team March 19, 2021 18:17
@modular-magician

This comment has been minimized.

@modular-magician

This comment has been minimized.

@melinath
Copy link
Member Author

/gcbrun - same reason

@modular-magician

This comment has been minimized.

@modular-magician

This comment has been minimized.

@melinath
Copy link
Member Author

/gcbrun for the same reason again

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 473 insertions(+))
Terraform Beta: Diff ( 7 files changed, 473 insertions(+))
TF Conversion: Diff ( 1 file changed, 22 insertions(+))

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=178114"

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccAccessContextManager|TestAccNotebooksInstance_notebookInstanceFullExample You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=178125"

@modular-magician
Copy link
Collaborator

Tests failed during RECORDING mode: TestAccNotebooksInstance_notebookInstanceFullExample Please fix these to complete your PR


func dataSourceGoogleKmsSecretAsymmetric() *schema.Resource {
return &schema.Resource{
ReadContext: dataSourceGoogleKmsSecretAsymmetricReadContext,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the first datasource/resource in TPG to use the new context-aware CRUD ops.

@google-cla
Copy link

google-cla bot commented Mar 22, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@mvanholsteijn
Copy link
Contributor

@googlebot I consent.

@melinath
Copy link
Member Author

melinath commented Apr 1, 2021

@mvanholsteijn - we ended up having to revert this. Would you be able to open a new PR to take a stab at fixing this issue? The error we're seeing is specifically:

[INFO] Instantiating Google Cloud KMS client for path on endpoint cloudkms.googleapis.com:443
[WARN] Error creating client kms: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

Followed by:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x1635352]

goroutine 883 [running]:
cloud.google.com/go/kms/apiv1.(*KeyManagementClient).AsymmetricDecrypt(0x0, 0x3769b50, 0xc000de4420, 0xc000de44e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
  .../pkg/mod/cloud.google.com/go@v0.78.0/kms/apiv1/key_management_client.go:962 +0x1f2
github.com/hashicorp/terraform-provider-google/google.dataSourceGoogleKmsSecretAsymmetricRead(0x3769b50, 0xc000de4420, 0xc0008e6280, 0x3101fe0, 0xc00041cc00, 0x479b960, 0x3769b50)
  .../src/github.com/terraform-providers/terraform-provider-google/google/data_source_google_kms_secret_asymmetric.go:105 +0x519
github.com/hashicorp/terraform-provider-google/google.dataSourceGoogleKmsSecretAsymmetricReadContext(0x3769b50, 0xc000de4420, 0xc0008e6280, 0x3101fe0, 0xc00041cc00, 0xc000ba8770, 0xc000f0d948, 0x410338)
  .../src/github.com/terraform-providers/terraform-provider-google/google/data_source_google_kms_secret_asymmetric.go:54 +0x57
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc0003aa480, 0x3769ae0, 0xc000d30440, 0xc0008e6280, 0x3101fe0, 0xc00041cc00, 0x0, 0x0, 0x0)
  .../pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.4.4/helper/schema/resource.go:297 +0x1ed
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).ReadDataApply(0xc0003aa480, 0x3769ae0, 0xc000d30440, 0xc000baa2a0, 0x3101fe0, 0xc00041cc00, 0xc00041cc00, 0xc000baa2a0, 0x0, 0x0)
  .../pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.4.4/helper/schema/resource.go:498 +0xfd
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadDataSource(0xc00137b560, 0x3769ae0, 0xc000d30440, 0xc000baa1c0, 0xc000d30440, 0x40dba5, 0x2e93760)
  .../pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.4.4/helper/schema/grpc_provider.go:1105 +0x4d6
github.com/hashicorp/terraform-plugin-go/tfprotov5/server.(*server).ReadDataSource(0xc000e15940, 0x3769b88, 0xc000d30440, 0xc0008983c0, 0xc000e15940, 0xc000b8aba0, 0xc000ec8ba0)
  .../pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.2.1/tfprotov5/server/server.go:247 +0xe5
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadDataSource_Handler(0x304b700, 0xc000e15940, 0x3769b88, 0xc000b8aba0, 0xc000de4300, 0x0, 0x3769b88, 0xc000b8aba0, 0xc0001c1000, 0x3ad)
  .../pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.2.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:416 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000c64c40, 0x37a6158, 0xc0011a1500, 0xc000172200, 0xc000c51650, 0x477f9f0, 0x0, 0x0, 0x0)
  .../pkg/mod/google.golang.org/grpc@v1.36.0/server.go:1217 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc000c64c40, 0x37a6158, 0xc0011a1500, 0xc000172200, 0x0)
  .../pkg/mod/google.golang.org/grpc@v1.36.0/server.go:1540 +0xd0c
google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000917cd0, 0xc000c64c40, 0x37a6158, 0xc0011a1500, 0xc000172200)
  .../pkg/mod/google.golang.org/grpc@v1.36.0/server.go:878 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
  .../pkg/mod/google.golang.org/grpc@v1.36.0/server.go:876 +0x1fd

projectOrg := getTestOrgFromEnv(t)
projectBillingAccount := getTestBillingAccountFromEnv(t)

projectID := "terraform-" + randString(t, 10)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a side note, this will also need to use tf-test- rather than terraform-

@mvanholsteijn
Copy link
Contributor

My implementation used the new, recommended Google Cloud KMS Client Library cloud.google.com/go/kms/apiv1. This library uses the gRPC transport and does not work with the HTTP client setup in the config.LoadAndValidate() method. I failed to pass in the credentials when creating a new KMS client and failed to check the result for nil. hence the crash.

Aside from the credentials, the LoadAndValidate() method also prepare the client with retry and timeout configuration for the HTTP client. As I am unsure how this would be done for the gRPC client, I am reverting the implementation back to the "google.golang.org/api/cloudkms/v1" library, which is also used by google_kms_secret.

@mvanholsteijn
Copy link
Contributor

Checkout my PR on hashicorp/terraform-provider-google-beta#3126. Do you want a PR on the magic modules?

@melinath
Copy link
Member Author

melinath commented Apr 6, 2021

@mvanholsteijn I've upstreamed the new changes and made a new PR against Magic Modules: #4664 - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants