Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(core): Templating error on custom-resource introduced with version 1.92 #13465

Closed
mzakarian opened this issue Mar 8, 2021 · 10 comments · Fixed by #11224 or #13544
Closed

(core): Templating error on custom-resource introduced with version 1.92 #13465

mzakarian opened this issue Mar 8, 2021 · 10 comments · Fixed by #11224 or #13544
Assignees
Labels
@aws-cdk/core Related to core CDK functionality bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@mzakarian
Copy link

mzakarian commented Mar 8, 2021

for (let index = 0; index < existingVpc.availabilityZones.length; index++) {
      let getEndpointIp = new AwsCustomResource(this, idPrefix + 'GetEndpointIp' + index, {
        policy: AwsCustomResourcePolicy.fromSdkCalls({resources: AwsCustomResourcePolicy.ANY_RESOURCE}),
        onUpdate: {
          action: 'describeNetworkInterfaces',
          service: 'EC2',
          parameters: {NetworkInterfaceIds: vpcendpoint.vpcEndpointNetworkInterfaceIds},
          physicalResourceId: PhysicalResourceId.of('DirectoryId'),
          outputPath: 'NetworkInterfaces.' + index + '.PrivateIpAddress'
        }
      })
      targetGroup.addTarget(new IpTarget(Token.asString(getEndpointIp.getResponseField('NetworkInterfaces.' + index + '.PrivateIpAddress'))));
    }

We are doing this, because we need to put an api gateway behind a load balancer (company policy). If there is a better solution to do this, feedback is welcome.

Reproduction Steps

this is the output with aws-cdk/core 1.91

...
  announceGetEndpointIp0D5B4E67A:
    Type: Custom::AWS
    Properties:
      ServiceToken:
        Fn::GetAtt:
          - AWS12345678
          - Arn
      Create:
        action: describeNetworkInterfaces
        service: EC2
        parameters:
          NetworkInterfaceIds:
            Fn::GetAtt:
              - announcevpcannouncevpcendpoint55991BB5
              - NetworkInterfaceIds
        physicalResourceId:
          id: DirectoryId
        outputPath: NetworkInterfaces.0.PrivateIpAddress
      Update:
        action: describeNetworkInterfaces
        service: EC2
        parameters:
          NetworkInterfaceIds:
            Fn::GetAtt:
              - announcevpcannouncevpcendpoint55991BB5
              - NetworkInterfaceIds
        physicalResourceId:
          id: DirectoryId
        outputPath: NetworkInterfaces.0.PrivateIpAddress
      InstallLatestAwsSdk: true
    DependsOn:
      - announceGetEndpointIp0CustomResourcePolicyC5EFFE47
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: announce-dev-local/announce-GetEndpointIp0/Resource/Default
...

this is the output with aws-cdk/core 1.92

...
  announceGetEndpointIp0D5B4E67A:
    Type: Custom::AWS
    Properties:
      ServiceToken:
        Fn::GetAtt:
          - AWS12345678
          - Arn
      Create:
        Fn::Join:
          - ""
          - - '{"action":"describeNetworkInterfaces","service":"EC2","parameters":{"NetworkInterfaceIds":"'
            - Fn::GetAtt:
                - announcevpcannouncevpcendpoint55991BB5
                - NetworkInterfaceIds
            - '"},"physicalResourceId":{"id":"DirectoryId"},"outputPath":"NetworkInterfaces.0.PrivateIpAddress"}'
      Update:
        Fn::Join:
          - ""
          - - '{"action":"describeNetworkInterfaces","service":"EC2","parameters":{"NetworkInterfaceIds":"'
            - Fn::GetAtt:
                - announcevpcannouncevpcendpoint55991BB5
                - NetworkInterfaceIds
            - '"},"physicalResourceId":{"id":"DirectoryId"},"outputPath":"NetworkInterfaces.0.PrivateIpAddress"}'
      InstallLatestAwsSdk: true
    DependsOn:
      - announceGetEndpointIp0CustomResourcePolicyC5EFFE47
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: announce-dev-local/announce-GetEndpointIp0/Resource/Default
...

What did you expect to happen?

After using cdk deploy we expected the same behavior as in version 1.91 with successful deployment.

What actually happened?

The error only occurs during a cdk deploy with version 1.92.

26/44 | 8:32:42 AM | CREATE_FAILED        | Custom::AWS                               | announce-GetEndpointIp2/Resource/Default (announceGetEndpointIp20B447BE7) Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.
	new CustomResource (/builds/marceldarvis/toolbox/services/announce-pipeline-modules-service/node_modules/@aws-cdk/core/lib/custom-resource.ts:119:21)
	\_ new AwsCustomResource (/builds/marceldarvis/toolbox/services/announce-pipeline-modules-service/node_modules/@aws-cdk/custom-resources/lib/aws-custom-resource/aws-custom-resource.ts:374:27)

Environment

  • CDK CLI Version :1.92
  • Framework Version:None
  • Node.js Version:v14.16.0
  • OS :linux
  • Language (Version):TypeScript (4.2.3)

Other

We assume that this issue is related to this change:


This is 🐛 Bug Report

@mzakarian mzakarian added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Mar 8, 2021
@github-actions github-actions bot added the @aws-cdk/core Related to core CDK functionality label Mar 8, 2021
@eladb
Copy link
Contributor

eladb commented Mar 9, 2021

I think this is actually an issue with how toJsonString() encodes non-string attributes.

The change in #13074 encodes the API requests as JSON strings so it will be possible to use tokens in e.g. map keys. The stack.toJsonString() uses Fn::Join in order to concatenate tokens into a single JSON string.

However, in this use case vpcendpoint.vpcEndpointNetworkInterfaceIds resolves to an Fn::GetAtt that returns array instead of a string (very rare for attributes) and Fn::Join cannot concatenate it.

@mergify mergify bot closed this as completed in #11224 Mar 10, 2021
mergify bot pushed a commit that referenced this issue Mar 10, 2021
Our previous implementation of `toJSON()` was quite hacky.

It replaced values inside the structure with objects that had a custom
`toJSON()` serializer, and then called `JSON.stringify()` on the result.

The resulting JSON would have special markers in it where the Token
values would be string-substituted back in.

It's actually easier and gives us more control to just implement
JSONification ourselves in a Token-aware recursive function.

This change has been split off from a larger, upcoming PR in order
to make the individual reviews smaller.

Incidentally also fixes #13465, as the type of encoded tokens is assumed to match
the type of the encoded value (e.g., a `string[]`-encoded token is assumed to
produce a list at deploy-time and so will not be quoted).


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@mzakarian
Copy link
Author

@rix0rrr @eladb thank you for resolving this issue ❤️

@jogold
Copy link
Contributor

jogold commented Mar 10, 2021

@eladb @rix0rrr I'm not sure I understand how is this issue fixed with the refactor of CloudFormationLang.toJSON()? You are still going to get a Fn::GetAtt returning an array after the change, no?

Just tested this:

image

image

@rix0rrr
Copy link
Contributor

rix0rrr commented Mar 11, 2021

🤦‍♂️

@jogold you are correct. I feel silly now, I was solving the wrong problem :(

@jogold
Copy link
Contributor

jogold commented Mar 11, 2021

@rix0rrr the only way to solve this right now for the user is this:

{
  NetworkInterfaceIds: [cdk.Token.asList(cdk.Fn.join(',', endpoint.vpcEndpointNetworkInterfaceIds))]
}

which will give the following correct CF:

{
  "Fn::Join": [
    "",
    [
      "{\"action\":\"describeNetworkInterfaces\",\"service\":\"EC2\",\"parameters\":{\"NetworkInterfaceIds\":[",
      {
        "Fn::Join": [
          ",",
          {
            "Fn::GetAtt": [
              "EndpointEEF1FD8F",
              "NetworkInterfaceIds"
            ]
          }
        ]
      },
      "]},\"physicalResourceId\":{\"id\":\"physical-od\"},\"outputPath\":\"NetworkInterfaces.0.PrivateIpAddress\"}"
    ]
  ]
}

@jogold
Copy link
Contributor

jogold commented Mar 11, 2021

@rix0rrr actually the code above doesn't work because the items in the vpcEndpointNetworkInterfaceIds aren't quoted.

This is very ugly but it works:

{
  NetworkInterfaceIds: [cdk.Token.asList(cdk.Stack.of(stack).toJsonString(cdk.Fn.join('","', endpoint.vpcEndpointNetworkInterfaceIds)))]
}
{
  "Fn::Join": [
    "",
    [
      "{\"action\":\"describeNetworkInterfaces\",\"service\":\"EC2\",\"parameters\":{\"NetworkInterfaceIds\":[\"",
      {
        "Fn::Join": [
          "\\\",\\\"",
          {
            "Fn::GetAtt": [
              "EndpointEEF1FD8F",
              "NetworkInterfaceIds"
            ]
          }
        ]
      },
      "\"]},\"physicalResourceId\":{\"id\":\"physical-od\"},\"outputPath\":\"NetworkInterfaces.0.PrivateIpAddress\"}"
    ]
  ]
}

@rix0rrr
Copy link
Contributor

rix0rrr commented Mar 11, 2021

Oh wow.

I hadn't even thought that far ahead. I was thinking we'll resort to CfnJson but you're right, this ugly hackery might also just work.

Not sure it's worth it though, as it's probably going to break in further edge cases.

@rix0rrr
Copy link
Contributor

rix0rrr commented Mar 11, 2021

Oh no it's actually fine. This will only trigger for CFN intrinsics which cannot be anything other than lists of strings anyway.

@rix0rrr
Copy link
Contributor

rix0rrr commented Mar 11, 2021

Doing this will turn [] -> [""] though :(

rix0rrr added a commit that referenced this issue Mar 11, 2021
The previous attempt at a fix missed one important case: the types
of the values involved in the `{ Fn::Join }` expression didn't actually
match up. They all needed to be strings, but the previous implentation
just dropped list-typed values in there.

Unfortunately, there is no way to do it correctly with just string
manipulation in CloudFormation (`{ Fn::Join }` etc), so we'll have
to resort to using a Custom Resource if we encounter list values.

Actually fixes #13465.
mergify bot pushed a commit that referenced this issue Mar 16, 2021
The previous attempt at a fix missed one important case: the types
of the values involved in the `{ Fn::Join }` expression didn't actually
match up. They all needed to be strings, but the previous implentation
just dropped list-typed values in there.

Unfortunately, there is no way to do it correctly with just string
manipulation in CloudFormation (`{ Fn::Join }` etc), so we'll have
to resort to using a Custom Resource if we encounter list values.

Actually fixes #13465.


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
hollanddd pushed a commit to hollanddd/aws-cdk that referenced this issue Mar 17, 2021
The previous attempt at a fix missed one important case: the types
of the values involved in the `{ Fn::Join }` expression didn't actually
match up. They all needed to be strings, but the previous implentation
just dropped list-typed values in there.

Unfortunately, there is no way to do it correctly with just string
manipulation in CloudFormation (`{ Fn::Join }` etc), so we'll have
to resort to using a Custom Resource if we encounter list values.

Actually fixes aws#13465.


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
hollanddd pushed a commit to hollanddd/aws-cdk that referenced this issue Mar 18, 2021
The previous attempt at a fix missed one important case: the types
of the values involved in the `{ Fn::Join }` expression didn't actually
match up. They all needed to be strings, but the previous implentation
just dropped list-typed values in there.

Unfortunately, there is no way to do it correctly with just string
manipulation in CloudFormation (`{ Fn::Join }` etc), so we'll have
to resort to using a Custom Resource if we encounter list values.

Actually fixes aws#13465.


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
hollanddd pushed a commit to hollanddd/aws-cdk that referenced this issue Aug 26, 2021
The previous attempt at a fix missed one important case: the types
of the values involved in the `{ Fn::Join }` expression didn't actually
match up. They all needed to be strings, but the previous implentation
just dropped list-typed values in there.

Unfortunately, there is no way to do it correctly with just string
manipulation in CloudFormation (`{ Fn::Join }` etc), so we'll have
to resort to using a Custom Resource if we encounter list values.

Actually fixes aws#13465.


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/core Related to core CDK functionality bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
4 participants