feat: Support AWS ECR Private Container Registry based on PostgreSQL

[jopemachine]

Partially fix #2337.

Test

ecr

Manually tested it using the following methods.

In this PR, the cr.backend.ai/stable/python:3.9-ubuntu20.04 image is used as a placeholder for testing.

Prerequisite

Create and configure the relevant users and policies appropriately through AWS IAM for testing this PR.

Then, generate the "access key" and store the access_key and secret_access_key in etcd.

---

1. Tag and push the cr.backend.ai/stable/python image to your AWS ECR package for testing.

❯ docker tag cr.backend.ai/stable/python:3.9-ubuntu20.04 <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<project>/python:3.9-ubuntu20.04
❯ docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<project>/python:3.9-ubuntu20.04

2. After filling in the required fields in the fixtures/manager/example-container-registries-aws-ecr.json file, execute the following command.

❯ ./backend.ai mgr fixture populate ./fixtures/manager/example-container-registries-aws-ecr.json

3. Insert the following value into the container_registry column of the groups table.

{
	"registry": "<aws_account_id>.dkr.ecr.<region>.amazonaws.com",
	"project": "<project>"
}

Note

Empty project values are not allowed.

4. Add <aws_account_id>.dkr.ecr.<region>.amazonaws.com value to allowed_docker_registries column of domains table using below command

❯ ./backend.ai admin domain update default --allowed-docker-registries=<aws_account_id>.dkr.ecr.<region>.amazonaws.com

Test scenarios

Verify that the container registry added in this PR is functioning based on several scenarios.

If there are additional scenarios that need testing, please leave a comment.

1. Image Rescan

Image rescanning should work through the following command.

❯ ./backend.ai mgr image rescan <aws_account_id>.dkr.ecr.<region>.amazonaws.com

2. Create a session from the pulled image

Run a session using the downloaded image.

❯ ./backend.ai session create <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<project>/python:3.9-ubuntu20.04

3. Commit the changes and push it to the container registry.

Commit the changes using the convert-to-image command and push them to the container registry.

❯ ./backend.ai session convert-to-image <session_id> test_imgname

∙ Request to commit Session(name or id: ce592e27-b90a-4859-92b1-360fdd6d8464)
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.02it/s]
✓ Session export process completed.

---

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue

---

📚 Documentation preview 📚: https://sorna--2624.org.readthedocs.build/en/2624/

---

📚 Documentation preview 📚: https://sorna-ko--2624.org.readthedocs.build/ko/2624/

[jopemachine]

• feat: Allow empty image name #2831
• feat: Replace image rescan command's --local flag with the local type ContainerRegistry record #2665
• feat: Support AWS ECR Private Container Registry based on PostgreSQL #2624 👈
• feat: Support AWS ECR Public Container Registry based on PostgreSQL #2623
• feat: Support GitLab Container Registry based on PostgreSQL #2622
• feat: Support GitHub Container Registry based on PostgreSQL #2621
• feat: Support docker image manifest v2 schema1 scanning #2815
• refactor: Base ContainerRegistry's scan_tag and implement MEDIA_TYPE_DOCKER_MANIFEST type handling #2620
• feat: Implement ID based client workflow to ContainerRegistry graphql API #2615
• feat: Add project column to the images table and refactor ImageRef related logics #2707
• feat: Migrate container registry config storage from Etcd to PostgreSQL #1917
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jopemachine and the rest of your teammates on Graphite

[kyujin-cho]

Merge activity

• Sep 30, 11:44 AM EDT: Graphite rebased this pull request after merging its parent, because this pull request is set to merge when ready.