apigateway: Multiple StepFunctionsIntegrations with options lead to "Property MethodResponses contains duplicate values"

[boenhoff]

Describe the bug

Using the StepFunctionsIntegration.startExecution() integration with API Gateway with cors options (see reproduction steps) works, as long as only one method has the integration with options.

If a second method uses the same options you will get the error
The stack named TestStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Property MethodResponses contains duplicate values., Property MethodResponses contains duplicate values. and in the cdk.out it generates multiple ResponseModels.

Expected Behavior

Multiple methods with the StepFunctionsIntegration.startExecution() creates methods with individual options.

Current Behavior

❌ Deployment failed: Error: The stack named TestStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Property MethodResponses contains duplicate values., Property MethodResponses contains duplicate values.
    at FullCloudFormationDeployment.monitorDeployment (/opt/homebrew/Cellar/aws-cdk/2.89.0/libexec/lib/node_modules/aws-cdk/lib/index.js:426:10236)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.deployStack2 [as deployStack] (/opt/homebrew/Cellar/aws-cdk/2.89.0/libexec/lib/node_modules/aws-cdk/lib/index.js:429:153307)
    at async /opt/homebrew/Cellar/aws-cdk/2.89.0/libexec/lib/node_modules/aws-cdk/lib/index.js:429:136985

The stack named TestStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Property MethodResponses contains duplicate values., Property MethodResponses contains duplicate values.

Reproduction Steps

• create a new CDK project dk init app --language typescript
• insert the following Code in the Stack
• deploy with cdk deploy --require-approval never

        const methodOptions = {
            methodResponses: [
                {
                    statusCode: '200',
                    responseParameters: {
                        'method.response.header.Access-Control-Allow-Origin': true,
                    }
                },
            ]
        }

        const integrationOptions = {
            integrationResponses: [
                {
                    responseParameters: {
                        'method.response.header.Access-Control-Allow-Origin': "'*'",
                    },
                    statusCode: '200'
                }
            ]
        }
        const api = new RestApi(this, 'api');
        const machine = new StateMachine(this, 'machine', {
            stateMachineType: StateMachineType.EXPRESS,
            definitionBody: DefinitionBody.fromChainable(new Pass(this, 'Pass'))
        })
        // uncomment the following line and comment out the line after that to make it running - but without the cors options
        // api.root.addMethod('POST', StepFunctionsIntegration.startExecution(machine))
        api.root.addMethod('POST', StepFunctionsIntegration.startExecution(machine, integrationOptions), methodOptions)
        api.root.addMethod('GET', StepFunctionsIntegration.startExecution(machine, integrationOptions), methodOptions)

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.89.0 (build 2ad6683)

Framework Version

No response

Node.js Version

v20.5.0

OS

macOS 13.5

Language

Typescript

Language Version

TypeScript (5.1.6)

Other information

No response

[pahud]

Looks like it returns duplicate ResponseModels for

 "ResponseModels": {
       "application/json": "Empty"
      },
      "StatusCode": "200"
     },

 "MethodResponses": [
     {
      "ResponseParameters": {
       "method.response.header.Access-Control-Allow-Origin": true
      },
      "StatusCode": "200"
     },
     {
      "ResponseModels": {
       "application/json": "Empty"
      },
      "StatusCode": "200"
     },
     {
      "ResponseModels": {
       "application/json": "Error"
      },
      "StatusCode": "400"
     },
     {
      "ResponseModels": {
       "application/json": "Error"
      },
      "StatusCode": "500"
     },
     {
      "ResponseModels": {
       "application/json": "Empty"
      },
      "StatusCode": "200"
     },
     {
      "ResponseModels": {
       "application/json": "Error"
      },
      "StatusCode": "400"
     },
     {
      "ResponseModels": {
       "application/json": "Error"
      },
      "StatusCode": "500"
     }
    ],

[rix0rrr]

I'm not even sure the user's original request can even work. I just looked at the default integrationResponses of the StepFunctionsIntegration, and it's very complicated, in a way you wouldn't want to have to write yourself:

{
      statusCode: '200',
      responseTemplates: {
        /* eslint-disable */
        'application/json': [
          '#set($inputRoot = $input.path(\'$\'))',
          '#if($input.path(\'$.status\').toString().equals("FAILED"))',
            '#set($context.responseOverride.status = 500)',
            '{',
              '"error": "$input.path(\'$.error\')",',
              '"cause": "$input.path(\'$.cause\')"',
            '}',
          '#else',
            '$input.path(\'$.output\')',
          '#end',
        /* eslint-enable */
        ].join('\n'),
      },
    },
    {
      /**
       * Specifies the regular expression (regex) pattern used to choose
       * an integration response based on the response from the back end.
       * In this case it will match all '4XX' HTTP Errors
       */
      selectionPattern: '4\\d{2}',
      statusCode: '400',
      responseTemplates: {
        'application/json': `{
            "error": "Bad request!"
          }`,
      },
    },
    {
      /**
       * Match all '5XX' HTTP Errors
       */
      selectionPattern: '5\\d{2}',
      statusCode: '500',
      responseTemplates: {
        'application/json': '"error": $input.path(\'$.error\')',
      },
    },

I'm pretty sure that as soon as you pass this:

        const integrationOptions = {
            integrationResponses: [
                { /* ... */ }
            ],
        };

        api.root.addMethod('POST', StepFunctionsIntegration.startExecution(machine, integrationOptions), methodOptions)

You completely override the built-in integrationOptions and you have to rebuild all that Velocity templating.

I have prepared 2 different PRs to take a stab at this:

• fix(apigateway): StepFunctions integration adds unmodifiable methodresponses #26720: stop StepFunctionsIntegration from adding default methodresponses so you can override them
• feat(apigateway): mutate methodresponses added by integrations #26718: make addMethodResponse merge fields for the same statusCode. But since the same thing doesn't happen for the integrationResponses (that still overrides) I'm not sure if this is actually going to be useful. It will help you declare the Access-Control-Allow-Origin header, but won't allow you to set a value for it without redefining all the integration options.

I'm open to making this work in a nice way because it seems useful, but I would like some design input from someone with API Gateway experience before making any choices here.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[redvex]

@boenhoff @pahud I don't think this is included in the latest shipped version. I'm facing the same issue and I'm quite struggle. Could you help me to use a version where this fix is included? I tried to point npm to install the package from github, but npm hang indefinitely.

[redvex]

@rix0rrr I'm facing the issue you addressed in your PR: https://github.com/aws/aws-cdk/pull/26720/files
Is it really not useful to anyone?
Without it the example in this issue doesn't work, for example.