Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#16400] Fix and simplify InternalIpDiffSuppress function #9423

Merged
merged 7 commits into from
Nov 16, 2023

Conversation

LucaPrete
Copy link
Member

As per #16400 the PR

Fix (and simplify) InternalIpDiffSuppress function to suppress drifts for IPv6 with and without netmask.

@modular-magician
Copy link
Collaborator

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will run automatically.

@melinath, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@LucaPrete
Copy link
Member Author

cc @rileykarson FYI

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 45 insertions(+), 13 deletions(-))
Terraform Beta: Diff ( 2 files changed, 45 insertions(+), 13 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3206
Passed tests 2855
Skipped tests: 325
Affected tests: 26

Action taken

Found 26 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeVpnTunnel_defaultTrafficSelectors|TestAccComputeVpnTunnel_vpnTunnelBetaExample|TestAccComputeVpnGateway_targetVpnGatewayBasicExample|TestAccComputeVpnTunnel_regionFromGateway|TestAccComputeVpnTunnel_vpnTunnelBasicExample|TestAccComputeRouterInterface_basic|TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate|TestAccComputeServiceAttachment_serviceAttachmentExplicitProjectsExample|TestAccComputeServiceAttachment_serviceAttachmentBasicExample|TestAccComputeBackendBucket_externalCdnLbWithBackendBucketExample|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisNoAutomateDnsExample|TestAccComputeGlobalForwardingRule_globalForwardingRuleInternalExample|TestAccComputeGlobalForwardingRule_externalHttpLbMigBackendCustomHeaderExample|TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisExample|TestAccComputeGlobalForwardingRule_externalTcpProxyLbMigBackendExample|TestAccComputeForwardingRule_forwardingRuleIpAddressIpv6|TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExampleUpdate|TestAccComputeForwardingRule_ip|TestAccComputeForwardingRule_forwardingRuleVpcPscExampleUpdate|TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExample|TestAccComputeForwardingRule_forwardingRuleVpcPscNoAutomateDnsExample|TestAccComputeForwardingRule_forwardingRuleRegionalHttpXlbExample|TestAccComputeForwardingRule_forwardingRuleVpcPscExample|TestAccComputeForwardingRule_forwardingRulePscRecreate|TestAccLoggingProjectSink_updatePreservesCustomWriter

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeBackendBucket_externalCdnLbWithBackendBucketExample[Debug log]
TestAccComputeGlobalForwardingRule_externalHttpLbMigBackendCustomHeaderExample[Debug log]
TestAccComputeGlobalForwardingRule_externalTcpProxyLbMigBackendExample[Debug log]
TestAccComputeForwardingRule_forwardingRuleIpAddressIpv6[Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalHttpXlbExample[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccComputeVpnTunnel_defaultTrafficSelectors[Error message] [Debug log]
TestAccComputeVpnTunnel_vpnTunnelBetaExample[Error message] [Debug log]
TestAccComputeVpnGateway_targetVpnGatewayBasicExample[Error message] [Debug log]
TestAccComputeVpnTunnel_regionFromGateway[Error message] [Debug log]
TestAccComputeVpnTunnel_vpnTunnelBasicExample[Error message] [Debug log]
TestAccComputeRouterInterface_basic[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentExplicitProjectsExample[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentBasicExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_internalLoadBalancing[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisNoAutomateDnsExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_globalForwardingRuleInternalExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExampleUpdate[Error message] [Debug log]
TestAccComputeForwardingRule_ip[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscExampleUpdate[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscNoAutomateDnsExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRulePscRecreate[Error message] [Debug log]
TestAccLoggingProjectSink_updatePreservesCustomWriter[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

Copy link
Member

@melinath melinath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple questions below.

I'm also not sure that the tests for non-IP addresses are correct - I believe they should expect a format like projects/project_id/regions/region/addresses/address-name rather than a resource reference (which would get expanded to some other value)

addr_netmask_new := strings.Split(new, "/")

// Check if old or new are IPs (with or without netmask)
addr_old := func() net.IP {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these being handled in closures? That's not a common pattern in this codebase.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}

// If old and new have a netmask compare them,
// otherwise always assume they are the same
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super familiar with IPv6 - why is this a safe assumption to make? (A link to documentation explaining this would be great!)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this is not safe and it's not documented anywhere but it's exactly what we're trying to solve with this patch. Please look at the bug we're referencing (#16400).

@melinath
Copy link
Member

melinath commented Nov 7, 2023

It looks like the unit tests are also currently failing - please make sure unit tests are passing locally before pushing a new commit.

@LucaPrete
Copy link
Member Author

LucaPrete commented Nov 8, 2023 via email

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 101 insertions(+), 32 deletions(-))
Terraform Beta: Diff ( 2 files changed, 101 insertions(+), 32 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3223
Passed tests 2882
Skipped tests: 328
Affected tests: 13

Action taken

Found 13 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate|TestAccComputeServiceAttachment_serviceAttachmentExplicitProjectsExample|TestAccComputeServiceAttachment_serviceAttachmentBasicExample|TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisNoAutomateDnsExample|TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisExample|TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExampleUpdate|TestAccComputeForwardingRule_forwardingRuleVpcPscExampleUpdate|TestAccComputeForwardingRule_ip|TestAccComputeForwardingRule_forwardingRuleVpcPscNoAutomateDnsExample|TestAccComputeForwardingRule_forwardingRuleVpcPscExample|TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExample|TestAccComputeForwardingRule_forwardingRulePscRecreate|TestAccDataprocJobIamPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeForwardingRule_ip[Debug log]
TestAccDataprocJobIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentExplicitProjectsExample[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentBasicExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisNoAutomateDnsExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExampleUpdate[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscExampleUpdate[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscNoAutomateDnsExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRulePscRecreate[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 99 insertions(+), 35 deletions(-))
Terraform Beta: Diff ( 2 files changed, 99 insertions(+), 35 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3223
Passed tests 2883
Skipped tests: 328
Affected tests: 12

Action taken

Found 12 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate|TestAccComputeServiceAttachment_serviceAttachmentExplicitProjectsExample|TestAccComputeServiceAttachment_serviceAttachmentBasicExample|TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisNoAutomateDnsExample|TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisExample|TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExampleUpdate|TestAccComputeForwardingRule_forwardingRuleVpcPscExampleUpdate|TestAccComputeForwardingRule_forwardingRulePscRecreate|TestAccComputeForwardingRule_forwardingRuleVpcPscExample|TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExample|TestAccComputeForwardingRule_forwardingRuleVpcPscNoAutomateDnsExample|TestAccDataprocClusterIamPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataprocClusterIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccComputeServiceAttachment_serviceAttachmentBasicExampleUpdate[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentExplicitProjectsExample[Error message] [Debug log]
TestAccComputeServiceAttachment_serviceAttachmentBasicExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisNoAutomateDnsExample[Error message] [Debug log]
TestAccComputeGlobalForwardingRule_privateServiceConnectGoogleApisExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExampleUpdate[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscExampleUpdate[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRulePscRecreate[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalSteeringExample[Error message] [Debug log]
TestAccComputeForwardingRule_forwardingRuleVpcPscNoAutomateDnsExample[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

@LucaPrete
Copy link
Member Author

@melinath I really don't understand why I see different results locally:

lucaprete-macbookpro:terraform-provider-google lucaprete$ make testacc TEST=./google/tpgresource TESTARGS='-run=TestOptionalPrefixSuppress'
TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google/tpgresource -v -run=TestOptionalPrefixSuppress -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestOptionalPrefixSuppress
--- PASS: TestOptionalPrefixSuppress (0.00s)
PASS
ok  	github.com/hashicorp/terraform-provider-google/google/tpgresource	1.138s
lucaprete-macbookpro:terraform-provider-google lucaprete$

Any idea please?

Copy link
Member

@melinath melinath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LucaPrete it looks like you're running TestOptionalPrefixSuppress - but the test that's failing is TestInternalIpDiffSuppress. Any chance that's the issue?

It looks like the failing VCR tests may be related to this change as well, but I'm not exactly sure how. The errors are like:

Error: Error creating ForwardingRule: googleapi: Error 400: Invalid value for field 'resource.IPAddress': ''. No IP address specified for Private Service Connect forwarding rule., invalid

@LucaPrete
Copy link
Member Author

TestInternalIpDiffSuppress

Arg, sorry... not sure how I haven't noticed it. I'll fix the tests and get back, hopefully soon.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 94 insertions(+), 32 deletions(-))
Terraform Beta: Diff ( 2 files changed, 94 insertions(+), 32 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3225
Passed tests 2888
Skipped tests: 329
Affected tests: 8

Action taken

Found 8 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeForwardingRule_forwardingRuleIpAddressIpv6|TestAccComputeGlobalForwardingRule_externalHttpLbMigBackendCustomHeaderExample|TestAccComputeForwardingRule_forwardingRuleRegionalHttpXlbExample|TestAccComputeForwardingRule_ip|TestAccComputeGlobalForwardingRule_externalTcpProxyLbMigBackendExample|TestAccComputeBackendBucket_externalCdnLbWithBackendBucketExample|TestAccDataprocClusterIamPolicy|TestAccHealthcareDatasetIamPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeForwardingRule_forwardingRuleIpAddressIpv6[Debug log]
TestAccComputeGlobalForwardingRule_externalHttpLbMigBackendCustomHeaderExample[Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalHttpXlbExample[Debug log]
TestAccComputeForwardingRule_ip[Debug log]
TestAccComputeGlobalForwardingRule_externalTcpProxyLbMigBackendExample[Debug log]
TestAccComputeBackendBucket_externalCdnLbWithBackendBucketExample[Debug log]
TestAccDataprocClusterIamPolicy[Debug log]
TestAccHealthcareDatasetIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

@LucaPrete
Copy link
Member Author

@melinath replies to all your comments and fixed the code/tests

Copy link
Member

@melinath melinath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM apart from a couple tweaks to comments

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 94 insertions(+), 32 deletions(-))
Terraform Beta: Diff ( 2 files changed, 94 insertions(+), 32 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3225
Passed tests 2896
Skipped tests: 329
Affected tests: 0

$\textcolor{green}{\textsf{All tests passed in REPLAYING mode.}}$
View the build log

@LucaPrete
Copy link
Member Author

@melinath done! Thanks for the review

@melinath
Copy link
Member

@LucaPrete could you make sure the changes are pushed?

@LucaPrete
Copy link
Member Author

@LucaPrete could you make sure the changes are pushed?

Now I did. Sorry again.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 95 insertions(+), 32 deletions(-))
Terraform Beta: Diff ( 2 files changed, 95 insertions(+), 32 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3230
Passed tests 2897
Skipped tests: 330
Affected tests: 3

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccDataprocJobIamPolicy|TestAccDataprocClusterIamPolicy|TestAccDataSourceGoogleServiceAccountJwt

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataprocJobIamPolicy[Debug log]
TestAccDataprocClusterIamPolicy[Debug log]
TestAccDataSourceGoogleServiceAccountJwt[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

Copy link
Member

@melinath melinath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM

@melinath melinath merged commit 06aa6a6 into GoogleCloudPlatform:main Nov 16, 2023
12 checks passed
davcen pushed a commit to davcen/gcp-magic-modules that referenced this pull request Nov 17, 2023
…dPlatform#9423)

* [#16400] Fix and simplify InternalIpDiffSuppress function

* Fixes as per PR comments

* Adding new tests for different netmasks

* fix test

* Fixes to unit tests

* Fixes per comments

* Comments fixes

---------

Co-authored-by: Luca Prete <lucaprete@google.com>
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Nov 28, 2023
…dPlatform#9423)

* [#16400] Fix and simplify InternalIpDiffSuppress function

* Fixes as per PR comments

* Adding new tests for different netmasks

* fix test

* Fixes to unit tests

* Fixes per comments

* Comments fixes

---------

Co-authored-by: Luca Prete <lucaprete@google.com>
jialei-chen pushed a commit to jialei-chen/magic-modules that referenced this pull request Nov 29, 2023
…dPlatform#9423)

* [#16400] Fix and simplify InternalIpDiffSuppress function

* Fixes as per PR comments

* Adding new tests for different netmasks

* fix test

* Fixes to unit tests

* Fixes per comments

* Comments fixes

---------

Co-authored-by: Luca Prete <lucaprete@google.com>
trodge pushed a commit to trodge/magic-modules that referenced this pull request Dec 8, 2023
…dPlatform#9423)

* [#16400] Fix and simplify InternalIpDiffSuppress function

* Fixes as per PR comments

* Adding new tests for different netmasks

* fix test

* Fixes to unit tests

* Fixes per comments

* Comments fixes

---------

Co-authored-by: Luca Prete <lucaprete@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants