(aws-elasticsearch): Vpc.fromLookup returns dummy VPC if the L2 elasticsearch.Domain availabilityZoneCount is set to 3

[bleish]

If the L2 elasticsearch.Domain construct is used with vpcOptions and zoneAwareness, and availabilityZoneCount is set to 3, trying to retrieve a VPC via ec2.Vpc.fromLookup fails silently and instead uses a "dummy" VPC with an id of "vpc-12345". The actual error thrown is "When providing vpc options you need to provide a subnet for each AZ you are using", but this is due to the dummy VPC only having 2 subnets instead of the required 3. Interestingly, if the availabilityZoneCount is set to 2, the vpc lookup works fine and doesn't get the dummy VPC. The actual VPC I'm using for this contains 4 private subnets (one for each availability zone in our VPC), but the issue occurs even if I limit the vpcOptions.subnets to 3. This occurs if you try to synth, deploy, or use any command that compiles the cloudformation template. If I instead use the L1 elasticsearch.CfnDomain construct with the same properties, the fromLookup works as expected.

Reproduction Steps

• The cdk.context.json must first be removed (no traces of previous Vpc.fromLookups).
• Use synth on a stack containing the code below

const vpc = ec2.Vpc.fromLookup(this, "VPC", {
    vpcId: "a-valid-vpcid"
});

var esSg = new ec2.SecurityGroup(this, "ESSecurityGroup", {
    vpc
});

new Domain(this, "ESDomain", {
    version: ElasticsearchVersion.V7_7,
    vpcOptions: {
        securityGroups: [esSg],
        subnets: vpc.privateSubnets
    },
    zoneAwareness: {
        enabled: true,
        availabilityZoneCount: 3
    }
});

What did you expect to happen?

The lookup should find the VPC and populate the cdk.context.json. The synth should successfully show the resource template with the correct subnets and values.

What actually happened?

An error is thrown, "When providing vpc options you need to provide a subnet for each AZ you are using" due to the VPC lookup silently failing and instead giving dummy data.

Environment

• CDK CLI Version: 1.78.0
• Framework Version:
• Node.js Version: 14.15.0
• OS: Windows 10 (10.0.18363 Build 18363)
• Language (Version): TypeScript (3.8.0)

Other

There are numerous open bug reports that are similar to this regarding Vpc.FromLookup returning dummy data, but none of them fit this exact scenario. For reference, here are a few that might be related:

• VPC shouldn't select 0 subnets in "dummy" mode
• [ec2] keep getting vpc-12345 from vpc lookup
• Looking up an unknown VPC yields false Id (DUMMY_VPC_PROPS vpc-12345)

---

This is 🐛 Bug Report

[iliapolo]

@bleish Thanks for reporting this.

To workaround this problem for now, you can temporarily remove the Domain definition from the application, run cdk synth, and then put it back in. This first synth will query the actual VPC details and store them in the cdk.context.json file, which will be used from now on, so that the dummy VPC will not be used.

[davidhessler]

This issue also presents itself when combining Vpc.fromLookup with CDKPipelines. Adding the entries in cdk.context.json to cdk.json will fix this. Though, that seems to be related to a nested stack issue. See #6115. Might be an issue with CDKPipelines though.

[automartin5000]

Seeing this is still an issue a year later with the OpenSearch construct. Seems like it should be an easy fix.

[argenstijn]

Same issue with VPC-12345. But not using elasticsearch ....

[zomgbre]

Same issue using OpenSearch Domain, however mine is:
["dummy1a","dummy1b"]
"When providing vpc options you need to provide a subnet for each AZ you are using."

[fredex42]

This is a weird one. Affecting me to, aws-cdk version 2.29.1; aws-cdk-lib version 2.29.1

[greenpau]

I too have the issue when availabilityZoneCount is set to 3 when using OpenSearch:

    zoneAwareness: { enabled: true, availabilityZoneCount: 3 },

[greenpau]

If I set the count to 2, then I get a different message:

Resource handler returned message: "Invalid request provided: You must specify exactly two subnets because you?ve set zone count to two. (Service: OpenSearch, Status Code: 400, Request ID: 10e96d82
-38ae-40b8-b135-a24441c7159b)" (RequestToken: 1cc362cf-0f7b-8446-35a1-fdec54c501ae, HandlerErrorCode: InvalidRequest)

Catch22 😞

[greenpau]

The way to overcome this is to iterate over the private subnets and then use subnetFilters. See below:

            const vpcSubnets: string[] = [];
            props.vpc.privateSubnets.forEach((subnet) => {
                vpcSubnets.push(subnet.subnetId);
            });

            const domain = new cdk.aws_opensearchservice.Domain(this, "Domain", {
                vpc: props.vpc,
                vpcSubnets: [
                    {
                        // subnetType: cdk.aws_ec2.SubnetType.PRIVATE_WITH_EGRESS,
                        // onePerAz: true,
                        subnetFilters: [cdk.aws_ec2.SubnetFilter.byIds(vpcSubnets)],
                    },
                ],

[alexbushong]

It looks like this was fixed by this PR: #22654
which was deployed with version 2.55.0: https://github.com/aws/aws-cdk/blob/main/CHANGELOG.v2.md#2550-2022-12-14

[Leo10Gama]

Closing this issue as it seems like things have been fixed. If the problem persists, feel free to open a new issue.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.