Kong cannot integrate with AWS Lambdas over HTTP

[merkle-rking]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

2.8.1, 3.0.0

Current Behavior

Kong hard-codes the URL generation for the endpoint to use when contacting AWS Lambda functions in the Kong AWS Lambda Plugin.

https://github.com/Kong/kong/blob/master/kong/plugins/aws-lambda/handler.lua#L294

In a recent, related PR, it was said:

since the request to AWS lambda is always https

But this is not strictly true, when AWS lambda is run locally through the AWS SAM start-lambda command, a thin HTTP layer (not an AWS Gateway API layer) is stood up in front of the lambdas so they can be hit locally. As far as I can tell, there is no way through AWS SAM parameters to start this endpoint up over HTTPS, it always uses HTTP locally.

In order to get around this, we've configured our lambda routes in our declarative kong.yml config to point back to Kong itself to perform SSL termination. This config is specifically for Kong 2.8.1 but I also tried Kong 3.0.0 with minor updates to handle the AWS host/region change and saw similar results.

  - name: myLambdaFunction
    paths: [ "/" ]
    methods: [ "GET" ]
    plugins:
    - name: aws-lambda
      config:
        invocation_type: RequestResponse
        aws_key: DUMMY_KEY
        aws_secret: DUMMY_SECRET
        log_type: None
        timeout: 60000
        keepalive: 60000
        function_name: myLambdaFunction
        host: localhost
        port: 8443
        forward_request_body: true
        forward_request_headers: true
        forward_request_method: true
        forward_request_uri: true
        is_proxy_integration: true

Note the port used is 8443 which is Kong itself. Then, we have Kong catch the request using an AWS lambda header and perform the actual request to our local AWS Lambda HTTP endpoint running on port 3001:

services:
- name: lambda-proxy
  url: http://host.docker.internal:3001/
  routes:
  - name: lambda_proxy
    path_handling: "v0"
    strip_path: true
    headers:
      content-type: ["application/x-amz-json-1.1"]

Not to conflate this issue, but I'm also dealing with a problem I haven't been able to solve yet that may be related to the way we're implementing this work-around. Somehow, Kong is not generating the X-Forwarded* headers when it proxies these requests. Our Lambda function depends on X-Forwarded-Proto and fails requests (though I can see that none of the X-Forwarded* headers are generated by Kong, not just X-Forwarded-Proto). I'm hoping that by allowing the plugin to natively use HTTP and removing our work-around will address that issue. We've confirmed that X-Forwarded* headers are properly generated when Kong is deployed in AWS. This header is not related to trusted IPs. Our work-around for this problem is to put the X-Forwarded-Proto header on our initial requests in development and Kong does correctly forward the header, it just doesn't generate it itself or the generated header is getting dropped somewhere inside Kong.

Expected Behavior

Kong should be able to talk directly to a Lambda running locally through sam local start-lambda without requiring work-arounds to handle the hard-coded HTTPS URI that Kong constructs.

Steps To Reproduce

1. Spin up Kong on a docker network
2. Define a Lambda + SAM template definition with AWS SAM's template.yaml to facilitate local development
3. Start a Lambda HTTP endpoint locally using:

sam local start-lambda --docker-network <YOUR DOCKER NETWORK RUNNING KONG>

4. Configure Kong with a route/service to point to your Lambda
5. Kong will exclusively generate SSL URIs for contacting the Lambda and the request will fail.