fix(integ-tests): use transformToString on API call response

[colifran]

This PR fixes an issue where our assertions handler will fail when logging the response of an API call if the response body is a Blob. Specifically, the following error is thrown:

Converting circular structure to JSON
    --> starting at object with constructor 'TLSSocket'
    |     property 'parser' -> object with constructor 'HTTPParser'
    --- property 'socket' closes the circle (RequestId: 50c1b6cd-47d2-494f-baf1-d22646cd4e5f)

The fix for this issue was to call the transformToString method on the response body. This is mentioned in aws-sdk-js-v3 UPGRADE.md as the solution for Lambda::Invoke. Its unclear why S3::GetObject isn't mentioned as well, but it works for that too.

However, instead of explicitly extracting the Body property from a response, we now recursively traverse the response and detect any such blob types that should be converted to strings - this should protect us against any other APIs that may exhibit this behavior.

transformToString is implemented as part of the SdkStreamMixin interface.

Included in this PR is a fix for aws-s3-deployment/test/integ.bucket-deployment-substitution.ts which was previously failing with the error shown above.

Co-authored by: @iliapolo
Co-authored by: @vinayak-kukreja
Co-authored by: @scanlonp

Closes #27114

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[iliapolo]

For anyone looking at this, i'll take over the PR now and address my own comments :)

[mrgrain]

Can you add unit tests for coerceResponse?

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 498ddeb
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[msambol]

@colifran I'm getting the following when running integration tests in aws-lambda-python-alpha. Still investigating but wondering if it's related to this PR.

2023-09-16T13:54:21.340Z	4a446e5c-4fc1-4c97-8cb7-e8316ee45075	INFO	TypeError: Cannot read properties of undefined (reading 'req')
    at AwsApiCallHandler.processEvent (/var/task/index.js:32693:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AwsApiCallHandler.handle (/var/task/index.js:32360:26)
    at async Runtime.handler (/var/task/index.js:32753:20)

This is directly after the following log line:

2023-09-16T13:54:19.940Z	4a446e5c-4fc1-4c97-8cb7-e8316ee45075	INFO	SDK request to Lambda.invoke with parameters {
    "FunctionName": "integ-lambda-python-poetr-myhandlerpython38withhas-YlEnyP5nCKSl"
}

[rix0rrr]

Hey @msambol,

That's worrying. Can you go into a bit of detail what test is triggering this problem?

[msambol]

@rix0rrr / @colifran I went back to main to make sure I wasn't missing something. I updated this line to force a change in the integration test (renamed my_handler_inline_with_hashes to my_handler_inline_foo) and got this:

The stack named poetryDefaultTestDeployAssertE9C9CB8F failed to deploy: CREATE_FAILED (The following resource(s) failed to create: [LambdaInvoke631dde0680edf7d2f0eea8d9b9c06c75, LambdaInvokefbdb1fc975f2f372fd64335aa07b46b7, LambdaInvoke26ce5a20d1ffee210d056cd2aa005172, LambdaInvokeead1d1e2cc2d2a843a514ffdc015b2c4, LambdaInvoke26497bb1c12c14ed3deada8d7d76b39e, LambdaInvoke8cbaf98b5be6e4f6f81f5f10b5b8dc89, LambdaInvoke6fd2d93b0d07a2b4220fae7c91786512, LambdaInvoke9a0beb4ea6cc38db92e9ff664c085292]. ): Response object is too long., Response object is too long., Response object is too long., Response object is too long., Response object is too long., Response object is too long.

Different error than I originally got but I think related? Still digging.

[msambol]

@rix0rrr / @colifran One issue at a time... In this line, I logged the size of responseBody and it is 13611 bytes for tests in integ.function.poetry.ts. Maximum size is 4096 bytes (see here).

[mrgrain]

@msambol FYI you can pass --force in to always run the integration test.

It's possible this has broken one of the existing tests, 'cause you are absolutely right: They don't get automatically re-run for changes to integ-tests-alpha.

If the response body is too large, we will have to set output paths to limit the response.

[msambol]

Happy to submit a PR with some guidance.