Support fallback uploads

[thchia]

This is kind of a big PR. I will do my best to explain the logical grouping of changes. I have added tests for the changes.

s3-static-assets

• Upload the fallback HTML pages to S3 (if present)

lambda-at-edge-compat

• Since previously we only used the default lambda to handle origin-requests, we always built up a new response object in this library (for SSR and generating JSON). Rightly so, this response was not allowed to have the "special" Cloudfront headers added to it.
• Now however, we are using the lambda to handle origin-responses too, which come with an existing response. If this existing response already has one or more of the special Cloudfront headers, we must preserve them. Therefore now if you call writeHead, it will keep those headers.

lambda-at-edge

• I have refactored the main handler into separate functions, one for handling origin-requests and another for handling origin-responses.

• Changes to the request handler:

  • In addition to forwarding HTML and public pages to S3, we also forward any data request and requests that match a fallback regex (from the prerender manifest). This is in case a previous request has already occurred and we have the HTML or JSON in S3 already. If they are not in S3, we handle it in the origin-response.
  • We no longer deal with data requests in the origin request. We assume the JSON is already in S3, and handle errors in the origin response.

• New response handler:

  • If the response from S3 is not 403, ignore (S3 returns 403 for item not found).
  • If it is a data request, calculate the JSON and HTML.
    • If it is an SSG data request, upload the JSON and HTML to S3.
    • Finally, return the JSON (i.e. it originated from a client side transition to a SSR page).
  • If it is not a data request, then it is a request for an SSG page with fallback that has not been visited before
    • Fetch the fallback HTML from S3
    • Return this as the new response.

serverless component

• Add the origin response config to the default cache behaviour and the _next/data/* pattern
• Update the lambda role to allow it to get and put objects into the specified S3 bucket.
  • Note that I have kept the ability to pass in a custom arn, but if you do so and you want the fallback behaviour, the associated role needs the requisite permissions.
  • Also note that one of the libraries used to update the IAM role will ignore changes if the arn is already there. So people may need to delete that role before the changes take effect.

[danielcondemarin]

Thanks for the PR @thchia ! I'll take a look this weekend at reviewing it and hopefully getting merged.

[danielcondemarin]

I've finished taking a look at this and great work btw.

Just a few things that can be improved in future,

• Data requests for getServerSideProps pages could be done at origin-request rather than origin-response. That would shave off a good time from response times.
• Ideally the policy to allow S3 uploads would be automatically added even when an ARN is given.

Having said that I think this is good to be merged 🚀

[danielcondemarin]

FYI - alpha release @sls-next/serverless-component@1.17.0-alpha.5

[thchia]

Thanks @danielcondemarin ! Yeah looks like I missed how we could deal with the SSR requests in the request handler.