(pipelines) Support docker image caching on CodeBuild

[danilofuchs]

(See also #11956)

Use Case

Building Docker images from scratch can take a long time on CI.

This blog post by AWS suggests we take advantage of image layer caching to skip native dependencies installation and other time consuming tasks.

(Specially in Node, installing dependencies can take several minutes)

#1776 provided the ability to pull the latest tag before building an image, and using the --cache-from parameter to reuse layer caching.

However, #5733 removed this ability, as all images are published to a single ECR repository

Proposed Solution

My workflow to make it work is this:

• Set assets-ecr-repository-name context to the project_name, so each project has its own repository. (This is a workaround and may be deprecated in the future)
• Before each build, login to ECR on my Codebuild machine
• Fetch the most recently pushed tag via AWS CLI
  • TAG=$(aws ecr describe-images --repository-name project_name --query 'sort_by(imageDetails,& imagePushedAt)[0].imageTags[0]')
• Docker Pull this tag
  • docker pull project_name:$TAG
• Run AWS CDK (will this use the correct layer caching? I'm not sure)

I'm not really sure what is the best way to simplify this process.

Other

#8606 could solve the issue by allowing workarounds with the --cache-from option.

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[eladb]

@danilofuchs are you using CodeBuild? See the last paragraph in the CDK pipelines design doc: https://github.com/aws/aws-cdk-rfcs/blob/master/text/0049-continuous-delivery.md#publishing

[shellscape]

We're not using CodeBuild and deploying a fargate service using @aws-cdk/aws-ecs-patterns and ApplicationLoadBalancedFargateService with DockerImageAsset. Having a "cache from" (and even "cache to") specifier is key to negating very long build times.

[Durgaprasad-Budhwani]

Any update on this issue?

[alex5207]

Anyone found a good solution to this?