Package size exceeding lambda limits

[emileten]

I tried deploying this package as is to AWS lambda following the procedure in infrastructure/aws -- namely, deploying the package as a zip file in S3 which is then pulled by the lambda. I am getting an error due to the size of the unzipped package exceeding the limit of 262 MB -- I am at 263 MB.

One solution could be to switch to a docker-container based lambda but this is going to make the application slower. Instead, we could make the redis dependencies optional.

I am also suggesting that we pin more aggressively the package versions so as to prevent package size increase in the future -- which is what caused this issue (in the first place this package wasn't exceeding the size limit, and I guess that since then some of our dependencies increased in size and we hadn't pinned them).

What do you think of the latter?

[vincentsarago]

The current dev deployment is blocked by this.

There could be three solutions:

• remove Redis Cache
• use EFS to store the package (https://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/)
• use diskcache instead of redis
  • use efs volume as storage
  • use lambda /tmp disk (512Mb) ⚠️ this will be per lambda and will be cleaned once the lambda gets deleted