Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update config.go and framework_config.go to use transport.Creds for all cases #7875

Merged
merged 4 commits into from
May 10, 2023

Conversation

andyrzhao
Copy link
Contributor

This PR fixes the issue in hashicorp/terraform-provider-google#14411 by ensuring we use transport.Creds for initializing the credentials/tokenSource in all applicable code-path. Using transport.Creds ensures that the underlying OAuth2 HTTPClient used for token exchange is properly configured for mTLS - the current version of the code in terraform bypasses this logic when calling the oauth2 google package APIs directly.

What is not included in this PR:

  1. Removing the custom tokenSource override logic - this is because Terraform persists the custom tokenSource for use in other locations such as for initializing big table grpc client as well as populating data source config for GKE clusters. The transport API for creating HTTPClient does not export the underlying tokenSource for re-use. (This support may be added in a future version of the API, which is currently being re-designed.)
  2. Simplifying the duplicated ADC logic in terraform. We will revisit this in the future.

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
  • Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

Update config.go and framework_config.go to use transport.Creds for all cases

@modular-magician
Copy link
Collaborator

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @SarahFrench, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

Your assigned reviewer will help review your code by:

  • Ensuring it's backwards compatible, covers common error cases, etc.
  • Summarizing the change into a user-facing changelog note.
  • Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.


@modular-magician modular-magician added the awaiting-approval Pull requests that needs reviewer's approval to run presubmit tests label May 3, 2023
@rileykarson rileykarson requested review from rileykarson and removed request for SarahFrench May 3, 2023 22:28
@rileykarson
Copy link
Member

@SarahFrench: I've been working w/ @andyrzhao internally, I'll grab this review!

Copy link
Member

@rileykarson rileykarson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool- these interfaces ultimately call the same methods we used to, preserving more metadata throughout. I can't think of a reason that this wouldn't be backwards-compatible.

Would it be easy to add a test that failed before but succeeds now, or is configuring making mtls calls difficult?

@@ -601,7 +601,7 @@ func GetCredentials(ctx context.Context, data ProviderModel, initialCredentialsO
return *creds
}

creds, err := googleoauth.CredentialsFromJSON(ctx, []byte(contents), clientScopes...)
creds, err := transport.Creds(ctx, option.WithCredentialsJSON([]byte(contents)), option.WithScopes(clientScopes...))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: transport.Creds ultimately calls out to CredentialsFromJSONWithParams, so this is almost certainly compatible: https://github.com/googleapis/google-api-go-client/blob/cf0df64489c5086a57b9156d9648fe8342b22408/internal/creds.go#LL108C1-L108C1

@@ -625,14 +625,12 @@ func GetCredentials(ctx context.Context, data ProviderModel, initialCredentialsO

tflog.Info(ctx, "Authenticating using DefaultClient...")
tflog.Info(ctx, fmt.Sprintf(" -- Scopes: %s", clientScopes))
defaultTS, err := googleoauth.DefaultTokenSource(context.Background(), clientScopes...)
creds, err := transport.Creds(context.Background(), option.WithScopes(clientScopes...))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: DefaultTokenSource calls FindDefaultCredentials and extracts the TokenSource from the credenetials (which we then use to reconstruct a credentials object...), Creds calls FindDefaultCredentials and returns the credentials.

@modular-magician modular-magician removed the awaiting-approval Pull requests that needs reviewer's approval to run presubmit tests label May 4, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 402 insertions(+), 405 deletions(-))
Terraform Beta: Diff ( 2 files changed, 459 insertions(+), 464 deletions(-))

@andyrzhao
Copy link
Contributor Author

Cool- these interfaces ultimately call the same methods we used to, preserving more metadata throughout. I can't think of a reason that this wouldn't be backwards-compatible.

Would it be easy to add a test that failed before but succeeds now, or is configuring making mtls calls difficult?

Thanks for the review Riley! So in terms of test coverage, adding a unit test (such as verifying that the resulting tokenSource has an oauth2 http client configured if mTLS is enabled) does not seem appropriate since we would be testing the internal behavior of transport.Creds which is subject to change. An integration test that can verify the end-to-end behavior would be more suitable - it would involve a test user account configured with CAA policy making a test call to GCP APIs. I'm not familiar enough with Terraform to make a call on how easy it would be to set-up such a test. From CBA team perspective, we do have plans in the back log to add automated E2E test coverage for all OS's and client tools.

I did manually verify the 2 code paths I touched with this PR (by using the application default credentials, as well as by specifying the GOOGLE_CREDENTIALS override), and both now works with CAA.

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Affected tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

Co-authored-by: Riley Karson <rileykarson@google.com>
@modular-magician modular-magician added awaiting-approval Pull requests that needs reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that needs reviewer's approval to run presubmit tests labels May 4, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 6 insertions(+), 10 deletions(-))
Terraform Beta: Diff ( 2 files changed, 6 insertions(+), 10 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Affected tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

Co-authored-by: Riley Karson <rileykarson@google.com>
@modular-magician modular-magician added awaiting-approval Pull requests that needs reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that needs reviewer's approval to run presubmit tests labels May 5, 2023
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 6 insertions(+), 10 deletions(-))
Terraform Beta: Diff ( 2 files changed, 6 insertions(+), 10 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 2710
Passed tests 2424
Skipped tests: 279
Affected tests: 7

Action taken

Found 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeFirewallPolicyRule_multipleRules|TestAccAlloydbCluster_missingLocation|TestAccAlloydbBackup_missingLocation|TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example|TestAccDataSourceGoogleFirebaseAndroidAppConfig|TestAccDataSourceAlloydbLocations_basic|TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

Tests passed during RECORDING mode:
TestAccAlloydbCluster_missingLocation[Debug log]
TestAccAlloydbBackup_missingLocation[Debug log]
TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example[Debug log]
TestAccDataSourceGoogleFirebaseAndroidAppConfig[Debug log]
TestAccDataSourceAlloydbLocations_basic[Debug log]
TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample[Debug log]

Tests failed during RECORDING mode:
TestAccComputeFirewallPolicyRule_multipleRules[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

@rileykarson rileykarson merged commit 1d360b4 into GoogleCloudPlatform:main May 10, 2023
shuyama1 added a commit that referenced this pull request May 15, 2023
ericayyliu pushed a commit to ericayyliu/magic-modules that referenced this pull request Jul 26, 2023
…for all cases (GoogleCloudPlatform#7875)

Co-authored-by: Riley Karson <rileykarson@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants