Easier container version tracking

[acdha]

The way the CloudWatch agent releases are managed is complicated for anyone trying to deploy the official containers. For example, if I look at https://github.com/aws/amazon-cloudwatch-agent/releases I see that the latest released version is v1.300044.0 but that version is not available on https://gallery.ecr.aws/cloudwatch-agent/cloudwatch-agent (the only mention of that value is 1.300044.0b793), the version tagged latest appears to be 1.300045.1b823, and the two stable tags point to older builds from two months ago.

This became more of an issue for us because we use ECS heavily and had deployed the CloudWatch container sidewards following the documentation. This worked great until the ECS team deployed a breaking change trying to make container digests immutable, causing attempts to autoscale new containers in a deployed service to fail once the hash which latest points to changes from what it was at the time of deployment (as was the case in our ECR pull-through cache).

I was attempting to work through this by pinning the versions for CloudWatch and X-Ray to specific releases (e.g. v1.300044.0). Obviously I can do that by hand but it would be nice if there was a documented way to get the current release (or, ideally, the stable release) from some combination of the GitHub and Public ECR metadata at the time of service deployment so my Terraform code could do something like determine that v1.300044.0 is the latest GitHub release and use the public ECR container tagged 1.300044.0 in the task definitions.

[sky333999]

Hey @acdha , thanks for raising the issue to us.
I think our Releases page hasnt been updated to reflect the recent 1.300045.1 release that went out last week - we will get that taken care of.

In terms of our Releases page showing v1.300044.0 vs our image tags having 1.300044.0b793, its kinda expected. We cut a release against a specific commit sha which gets tagged with something like v1.300044.0, but when we build and publish our artifacts, we suffix a build number such as b1234. This is mainly so we can push a new build if there is any issue specifically with our build pipelines - i.e. the code is still pointing the same tag/commit, but we might re-build the artifacts which will then get a new build number, b5678. In reality, its very uncommon for more than one builds for the same version.

Coming to your issue, rather than relying on multiple sources such as GH Releases and Public ECR, it might be better to just rely on the latter.
Could you inspect the latest or stable image and then find the corresponding tagged image with the same matching digest?
Something like:

dig=$(docker image inspect public.ecr.aws/cloudwatch-agent/cloudwatch-agent:latest | jq -r '.[].RepoDigests[]' | awk -F@ '{print $2}')
curl -s 'https://api.us-east-1.gallery.ecr.aws/describeImageTags' --data-raw '{"registryAliasName":"cloudwatch-agent","repositoryName":"cloudwatch-agent"}' | jq -r '.imageTagDetails.[] | select(.imageDetail.imageDigest == "'$dig'") | .imageTag'

[acdha]

dig=$(docker image inspect public.ecr.aws/cloudwatch-agent/cloudwatch-agent:latest | jq -r '.[].RepoDigests[]' | awk -F@ '{print $2}')
curl -s 'https://api.us-east-1.gallery.ecr.aws/describeImageTags' --data-raw '{"registryAliasName":"cloudwatch-agent","repositoryName":"cloudwatch-agent"}' | jq -r '.imageTagDetails.[] | select(.imageDetail.imageDigest == "'$dig'") | .imageTag'

Thanks for the idea. I was hoping to avoid introducing the Docker stack as a dependency because we don't otherwise use it but Skopeo is a single binary and that works portably:

$ curl -s 'https://api.us-east-1.gallery.ecr.aws/describeImageTags' --data-raw '{"registryAliasName":"cloudwatch-agent","repositoryName":"cloudwatch-agent"}' | jq -r '.imageTagDetails[] | select(.imageTag != "latest" and .imageDetail.imageDigest == "'$(skopeo inspect docker://public.ecr.aws/cloudwatch-agent/cloudwatch-agent:latest --no-tags -f "{{.Digest}}")'") | .imageTag' -r
1.300046.0b833

[sky333999]

Resolving this for now. Please feel free to reopen if you have any further questions.