Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(opensearch): Don't throw a Az count mismatch for imported VPCs #22654

Merged
merged 17 commits into from
Dec 12, 2022
Merged

fix(opensearch): Don't throw a Az count mismatch for imported VPCs #22654

merged 17 commits into from
Dec 12, 2022

Conversation

stijnbrouwers
Copy link
Contributor

@stijnbrouwers stijnbrouwers commented Oct 26, 2022

Solving issue #22651

Creating a domain fails for imported vpc/subnets when zone awareness is enabled and the cdk context is cleared.
When the CDK context is already retrieved and the VPC is in the context, the deployment works.

This is due to the fact that when there is no context yet, the subnet count is always 0. That's why I decided to disable it. If it's not correct, it will fail when applying the CloudFormation template.


All Submissions:

Adding new Unconventional Dependencies:

  • This PR adds new unconventional dependencies following the process described here

New Features

  • Have you added the new feature to an integration test?
    • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Oct 26, 2022

@github-actions github-actions bot added beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK p2 labels Oct 26, 2022
@aws-cdk-automation aws-cdk-automation requested a review from a team October 26, 2022 14:37
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

@stijnbrouwers stijnbrouwers changed the title bugfix(opensearch): Don't throw a Az count mismatch (imported VPC) fix(opensearch): Don't throw a Az count mismatch for imported VPCs Oct 26, 2022
Copy link
Contributor

@TheRealAmazonKendra TheRealAmazonKendra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a succeeding build and the PR linter to pass before we can provide a meaningful review.

@aws-cdk-automation aws-cdk-automation dismissed their stale review October 26, 2022 16:42

✅ Updated pull request passes all PRLinter validations. Dissmissing previous PRLinter review.

@stijnbrouwers
Copy link
Contributor Author

We need a succeeding build and the PR linter to pass before we can provide a meaningful review.

Thanks, It should be OK now. I was still figuring out how to work with the integration tests :-)

@mergify mergify bot dismissed TheRealAmazonKendra’s stale review November 14, 2022 10:21

Pull request has been modified.

Copy link
Contributor Author

@stijnbrouwers stijnbrouwers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linter and build should be OK now

@Naumel
Copy link
Contributor

Naumel commented Nov 14, 2022

The linter and build should be OK now

The build is not...

@aws-cdk/aws-opensearchservice:   NEW        integ.opensearch.imported-vpc 0.913s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.cognitodashboardsauth 1.893s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.vpc 2.063s
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/aws-ec2" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/aws-ec2" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/aws-elasticloadbalancingv2" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/aws-elasticloadbalancingv2" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/aws-globalaccelerator" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/aws-globalaccelerator" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/core" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/core" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "constructs" at "^10.0.0" means you should take a "devDependency" on "constructs" at "10.0.0" (found "undefined")
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.advancedsecurity 3.694s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.ultrawarm 3.68s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.unsignedbasicauth 3.838s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch 3.957s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.custom-kms-key 3.949s
@aws-cdk/aws-opensearchservice: Snapshot Results: 
@aws-cdk/aws-opensearchservice: Tests:    1 failed, 8 total
@aws-cdk/aws-opensearchservice: Error: Some tests failed!
@aws-cdk/aws-opensearchservice: To re-run failed tests run: yarn integ-runner --update-on-failed
@aws-cdk/aws-opensearchservice:     at main (/codebuild/output/src720397436/src/github.com/aws/aws-cdk/packages/@aws-cdk/integ-runner/lib/cli.js:131:19)
@aws-cdk/aws-opensearchservice: Error: integ-runner exited with error code 1
@aws-cdk/aws-opensearchservice: Tests failed. Total time (19.1s) | /codebuild/output/src720397436/src/github.com/aws/aws-cdk/node_modules/jest/bin/jest.js (14.4s) | integ-runner (4.7s)
@aws-cdk/aws-opensearchservice: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@aws-cdk/aws-opensearchservice: error Command failed with exit code 1.
@aws-cdk/aws-opensearchservice: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
@aws-cdk/aws-autoscaling:   UNCHANGED  integ.amazonlinux2 3.625s
@aws-cdk/aws-opensearchservice: error Command failed with exit code 1.

@stijnbrouwers
Copy link
Contributor Author

stijnbrouwers commented Nov 16, 2022

The linter and build should be OK now

The build is not...

@aws-cdk/aws-opensearchservice:   NEW        integ.opensearch.imported-vpc 0.913s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.cognitodashboardsauth 1.893s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.vpc 2.063s
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/aws-ec2" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/aws-ec2" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/aws-elasticloadbalancingv2" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/aws-elasticloadbalancingv2" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/aws-globalaccelerator" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/aws-globalaccelerator" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "@aws-cdk/core" at "0.0.0" means you should take a "devDependency" on "@aws-cdk/core" at "0.0.0" (found "undefined")
@aws-cdk/aws-globalaccelerator-endpoints: warning JSII6: A "peerDependency" on "constructs" at "^10.0.0" means you should take a "devDependency" on "constructs" at "10.0.0" (found "undefined")
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.advancedsecurity 3.694s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.ultrawarm 3.68s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.unsignedbasicauth 3.838s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch 3.957s
@aws-cdk/aws-opensearchservice:   UNCHANGED  integ.opensearch.custom-kms-key 3.949s
@aws-cdk/aws-opensearchservice: Snapshot Results: 
@aws-cdk/aws-opensearchservice: Tests:    1 failed, 8 total
@aws-cdk/aws-opensearchservice: Error: Some tests failed!
@aws-cdk/aws-opensearchservice: To re-run failed tests run: yarn integ-runner --update-on-failed
@aws-cdk/aws-opensearchservice:     at main (/codebuild/output/src720397436/src/github.com/aws/aws-cdk/packages/@aws-cdk/integ-runner/lib/cli.js:131:19)
@aws-cdk/aws-opensearchservice: Error: integ-runner exited with error code 1
@aws-cdk/aws-opensearchservice: Tests failed. Total time (19.1s) | /codebuild/output/src720397436/src/github.com/aws/aws-cdk/node_modules/jest/bin/jest.js (14.4s) | integ-runner (4.7s)
@aws-cdk/aws-opensearchservice: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@aws-cdk/aws-opensearchservice: error Command failed with exit code 1.
@aws-cdk/aws-opensearchservice: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
@aws-cdk/aws-autoscaling:   UNCHANGED  integ.amazonlinux2 3.625s
@aws-cdk/aws-opensearchservice: error Command failed with exit code 1.

@Naumel
Sorry, for that, I thought this was working...
Anyway, It should be fixed now!

@mergify mergify bot dismissed mrgrain’s stale review December 10, 2022 12:58

Pull request has been modified.

@mrgrain mrgrain self-assigned this Dec 10, 2022
@mrgrain
Copy link
Contributor

mrgrain commented Dec 12, 2022

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Dec 12, 2022

update

✅ Branch has been successfully updated

Copy link
Contributor

@mrgrain mrgrain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code is looking good now and thanks for adding the integ test.

However the integ test does not appear to deploy successfully. Did you run it?

Context lookups have been disabled. Make sure all necessary context is already in 'cdk.context.json' by running 'cdk synth' on a machine with sufficient AWS credentials and committing the result. Missing context keys: 'vpc-provider:account=ACCOUNT:filter.tag:Name=my-vpc-name:region=us-east-1:returnAsymmetricSubnets=true'
  FAILED     integ.opensearch.imported-vpc-cdk-integ-opensearch-vpc-test/DefaultTest (undefined/us-east-1) 10.216s
      Integration test failed: Error: Command exited with status 1

Given that the repro steps for the issue includes "clear the context" this error seems related?!

@mergify mergify bot dismissed mrgrain’s stale review December 12, 2022 10:02

Pull request has been modified.

@stijnbrouwers
Copy link
Contributor Author

@mrgrain

Yes, I ran it and it used to work
image

After applying the app dependency change however, it failed again...
It is not clear what to do here.
I tried adding

/// !cdk-integ * pragma:enable-lookups

Which you removed (but also doesn't seem to do anything at the moment)
and I tried setting

"enableLookups": true

in integ.json.
But again, to no avail.

The unit test NEEDS to do a lookup, if the VPC is already in the context the bug does not occur.
So if we want to trigger the issue we need to have an empty context and do a lookup but the integration test framework does not seem to support this scenario.

@mrgrain
Copy link
Contributor

mrgrain commented Dec 12, 2022

Thanks for the detailed response @stijnbrouwers I think I misunderstood the source of the issue. Apologies for this.
If I re-phrase the test scenario in my own words, I think this needs to happen:

  • We need an exiting Stack with a VPC
  • Then we need to do a lookup, forced by the empty context

I'm not sure we can actually create this order in a single deployment (which is was the integ framework does). Because any lookups would happen before the Stack with the VPC would be deployed as well, thus failing the lookup. I'll need to think a bit more about this. This might be okay with a unit test only.

@stijnbrouwers
Copy link
Contributor Author

@mrgrain thank you very much for your quick response.
And that's 100% correct what you are saying.

The issue that I fixed is:
Whenever the opensearch is created, it does a sanity check to see that the minimum or provided availabilityZone count is present in the VPC subnets. However, when the VPC is provided by lookup, the subnet array is still empty when the check is being run, and thus always returns 0.

To mitigate this, I skip the check whenever one of the subnets has the isPendingLookup-property set to "true".

Should the minimum amount of availability zones not be present, it will fail upon creation of the actual resource in AWS instead of upon creation of the cloudformation template.

That should give a little insight into why I need to do the lookup in the integration test. Another solution would be to just always drop the sanity check and let AWS complain upon resource creation, but I liked this better :-)

@mrgrain
Copy link
Contributor

mrgrain commented Dec 12, 2022

I'm with you now and I agree with the assessment to skip the check in case the lookup is pending. We don't need an integ test for this, because it's not possible to test this with the current setup. Let's remove the test and get it merged.

@mrgrain mrgrain added the pr-linter/exempt-integ-test The PR linter will not require integ test changes label Dec 12, 2022
@stijnbrouwers
Copy link
Contributor Author

@mrgrain , great thanks!
I removed the integ test.

@mergify
Copy link
Contributor

mergify bot commented Dec 12, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: dafd14d
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 6a28b7f into aws:main Dec 12, 2022
@mergify
Copy link
Contributor

mergify bot commented Dec 12, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@stijnbrouwers
Copy link
Contributor Author

@mrgrain thank you very much for your support!

brennanho pushed a commit to brennanho/aws-cdk that referenced this pull request Jan 20, 2023
…ws#22654)

Solving issue aws#22651 

Creating a domain fails for imported vpc/subnets when zone awareness is enabled and the cdk context is cleared.
When the CDK context is already retrieved and the VPC is in the context, the deployment works.

This is due to the fact that when there is no context yet, the subnet count is always 0. That's why I decided to disable it. If it's not correct, it will fail when applying the CloudFormation template.

----

### All Submissions:

* [X] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
brennanho pushed a commit to brennanho/aws-cdk that referenced this pull request Feb 22, 2023
…ws#22654)

Solving issue aws#22651 

Creating a domain fails for imported vpc/subnets when zone awareness is enabled and the cdk context is cleared.
When the CDK context is already retrieved and the VPC is in the context, the deployment works.

This is due to the fact that when there is no context yet, the subnet count is always 0. That's why I decided to disable it. If it's not correct, it will fail when applying the CloudFormation template.

----

### All Submissions:

* [X] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK p2 pr-linter/exempt-integ-test The PR linter will not require integ test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants