[pipelines] Metadata errors should fail synthesis (on stacks deployed in the pipeline)

[J11522]

This issue is similar to #5594 however, occurs without nested stacks.
Instead, I have a cross-account CdkPipeline setup according to the minimal example from the documentation.
My Stack consists only of a simple VPC.

When deploying, the CloudFormation template in the target account still contains the dummy1a, dummy1b etc. AZs.
Finally, these are reported as invalid and my stack is rolled back.

Reproduction Steps

pipeline.addApplicationStage(new MyApplication(this, 'Dev', {
      env: {
        account: '11111111111',
        region: 'eu-west-1',
      }
    }));

Where MyApplication consist of a single stack with only new VPC(this, 'VPCName')

Error Log

Value (dummy1a) for parameter availabilityZone is invalid. Subnets can currently only be created in the following availability zones: eu-west-1a, eu-west-1b, eu-west-1c. (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue

Environment

• CLI Version : 1.54.0
• Framework Version: 1.54.0
• Node.js Version: 14.4.0
• OS :
• Language (Version): TypeScript (3.8.3)

Other

---

This is 🐛 Bug Report

[ericzbeard]

Pipelines does not currently support lookups.

(From the blog)

Developer preview limitations
CDK Pipelines is currently in developer preview. AWS is working to lift some limitations, but you should keep the following in mind:

No context queries – Context queries are not supported. That means that Vpc.fromLookup() and other functions like it don’t work.

[J11522]

Thanks for the pointer.
In case someone else encounters this, have a look a the context section: here
An alternative could be to do a dynamic lookup in the build/synth phase and populate the context there.

Do you propose to close this issue as this is a know limitation?

[ericzbeard]

I changed the title and made it a feature request.

[njdancer]

@J11522 I might be missing something. Does that page outline a workaround or just additional info around context and how to get the available AZ's out of band. The only way I seem to be able to workaround this is by overriding the availabilityZones member inside my Stack subclass.

[J11522]

@njdancer You are right, this is not a workaround but rather an offline solution.
An alternative would be to perform this lookup yourself via the AWS SDK, but then you could run into issues with the cross account setup.

[moshir]

What's the current/recommended way to create a VPC in a pipeline ?

[moshir]

@J11522 I might be missing something. Does that page outline a workaround or just additional info around context and how to get the available AZ's out of band. The only way I seem to be able to workaround this is by overriding the availabilityZones member inside my Stack subclass.

This does not seem to work, even when i override the Stack #.availability_zones(self), it ends up with dummy values :

class TheStack(core.Stack):
    def availability_zones(self) -> typing.List[str]:
        return ['eu-west-1a', 'eu-west-1b']

    def __init__(self, scope, id, **kwargs):
        super().__init__(
            scope,
            id,
            **kwargs
        )
            vpc = ec2.Vpc(self, "VPC",
                           max_azs=2,
                           cidr="172.31.0.0/16",
                           subnet_configuration=[ec2.SubnetConfiguration(
                               subnet_type=ec2.SubnetType.PUBLIC,
                               name="Public",

                               cidr_mask=20
                           ), ec2.SubnetConfiguration(
                               subnet_type=ec2.SubnetType.PRIVATE,
                               name="Private",
                               cidr_mask=24
                           )
                           ],
                           nat_gateways=1,
                           )

The output stack will still have dummy1a in the subnet AvailabilityZone:

    "VPCPublicSubnet1SubnetB4246D30": {
      "Type": "AWS::EC2::Subnet",
      "Properties": {
        "CidrBlock": "172.31.0.0/20",
        "VpcId": {
          "Ref": "VPCB9E5F0B4"
        },
        "AvailabilityZone": "dummy1a",

[J11522]

@moshir Try setting the AZs in your context.
You can do this through the context file very easily:

{
  "availability-zones:account=<your account id>:region=eu-west-1": [
    "eu-west-1a",
    "eu-west-1b",
    "eu-west-1c"
  ]
}

[arpowers]

There needs to be more clarity around how cdk.context.json works and what values are needed if we have to edit it directly.

How did you figure out what to add to that file? @J11522

[J11522]

@arpowers To be honest, I can't recall how I figured this out. I guess looking at code and documentation helped in the end.

[jguice]

If you run cdk deploy from your local, it should populate the cdk.context.json file with AZ info you can use as a template for additional regions, etc.

[victoravr]

Pipelines does not currently support lookups.

(From the blog)

Developer preview limitations
CDK Pipelines is currently in developer preview. AWS is working to lift some limitations, but you should keep the following in mind:

No context queries – Context queries are not supported. That means that Vpc.fromLookup() and other functions like it don’t work.

Are there any plans to support lookups like @aws-cdk/aws-ec2.Vpc.fromLookup() in @aws-cdk/pipelines anytime soon?

[SoManyHs]

Thanks @J11522 for the tip, adding that totally worked for me! https://github.com/SoManyHs/cdkpipelines-demo/blob/main/cdk.json#L8-L11

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[rix0rrr]

synth needs to be able check the manifests of selected stacks for errors. Normally deploy would do that, but since we're not using that, we don't see the errors right now.

[rix0rrr]

What's happening is something has failed during synthesis, but the synthesis CodeBuild step is not failing and that's why you're seeing dummy AZs in your template

[rix0rrr]

For example, if I run cdk synth -v I will see something like this in the log:

Setting "availability-zones:account=111111111:region=us-east-1" context to {"$providerError":"Need to perform AWS calls for account 111111111, but no credentials have been configured.","$dontSaveContext":true}

The mistake here is that this error is not failing the Pipelines synthesis, which it probably should.

The error itself is due to some kind of misconfiguration.

[lengebre]

@J11522 I might be missing something. Does that page outline a workaround or just additional info around context and how to get the available AZ's out of band. The only way I seem to be able to workaround this is by overriding the availabilityZones member inside my Stack subclass.

This does not seem to work, even when i override the Stack #.availability_zones(self), it ends up with dummy values :

class TheStack(core.Stack):
    def availability_zones(self) -> typing.List[str]:
        return ['eu-west-1a', 'eu-west-1b']

    def __init__(self, scope, id, **kwargs):
        super().__init__(
            scope,
            id,
            **kwargs
        )
            vpc = ec2.Vpc(self, "VPC",
                           max_azs=2,
                           cidr="172.31.0.0/16",
                           subnet_configuration=[ec2.SubnetConfiguration(
                               subnet_type=ec2.SubnetType.PUBLIC,
                               name="Public",

                               cidr_mask=20
                           ), ec2.SubnetConfiguration(
                               subnet_type=ec2.SubnetType.PRIVATE,
                               name="Private",
                               cidr_mask=24
                           )
                           ],
                           nat_gateways=1,
                           )

The output stack will still have dummy1a in the subnet AvailabilityZone:

    "VPCPublicSubnet1SubnetB4246D30": {
      "Type": "AWS::EC2::Subnet",
      "Properties": {
        "CidrBlock": "172.31.0.0/20",
        "VpcId": {
          "Ref": "VPCB9E5F0B4"
        },
        "AvailabilityZone": "dummy1a",

This will allow you to override the default availability zones.

@property
def availability_zones(self):
    return ['us-east-1d', 'us-east-1e']

def __init__(self, scope: core.Construct, construct_id: str, **kwargs) -> None:
    super().__init__(scope, construct_id, **kwargs)

    # The code that defines your stack goes here
    vpc = ec2.Vpc(
        self, "TestVpc",
        max_azs=2,
        cidr="10.0.0.0/16",
    )

[rix0rrr]

Should probably be an extension to the cdk metadata command?

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.