no cloud agents: aws

[dustymabe]

In #12 we decided that we'd like to try to not ship cloud agents. This ticket will document investigation and strategy for shipping without a cloud agent on the aws ec2 cloud platform.

See also #41 for a discussion of how to ship cloud specific bits using ignition.

[bgilbert]

AFAIK AWS doesn't use an agent.

[dustymabe]

Sounds good to me. I opened all of these tickets so we could explicitly document the state for each one. We can close this issue with a PR to the design doc.

[oliwer]

AWS has the ECS Container agent which is required if you want to be part of an ECS cluster. But I guess most people would prefer to use Amazon Linux for that.

Then there is the awscli, which can easily be installed as a container. Except in one case: if you want to use the AWS ECR Registry to host your images (which is a good idea since AWS SLA is way better than DockerHub's) you need to login to to the registry before you can pull images. And to login you need the awscli :( To solve this problem, there are 2 solution : use the unofficial ecr-credential-helper which you need to download at startup, or hack your own credential helper in bash+curl+openssl, and include it in your Ignition file.

[finchd]

AWS EC2 relies on cloud-init, DHCP, and udev rules for things like hot-attaching additional ENIs (NICs) and EBS (/dev/sd*, /dev/nvme*) while running. I'm personally a little fuzzy on whether custom udev rules were ever used, but I thought there were.

The ECS Agent is relatively easy to deliver by UserData at launch-time... but only with cloud-init, which to my knowledge still relies on Python2.

Aside: (Image config) Typically, booting relies on there only being one disk with one bootable partition, multiple disks with bootable partitions have observable pseudo-random startup /dev/ ordering. I've seen images with UUIDs or labels but they aren't common.

[ajeddeloh]

AWS does not rely on cloud-init, you just need something to do configuration which on most OSs is cloud-init. For us it will be Ignition. The udev rules can also be shipped in the image (since they'll only apply to amazon hardware). That's what we do on Container Linux today.

I'm not knowledgeable about the ECS agent but that might be a good candidate to leave out or containerize. We don't ship it on container linux today.

Booting with multiple bootable disks shouldn't be supported. We're planning on doing what CL does with an image that can boot from either uefi or bios. We use labels to get around the /dev ordering issue.

[dustymabe]

The udev rules can also be shipped in the image (since they'll only apply to amazon hardware). That's what we do on Container Linux today.

Are we already doing that in FCOS? Should we identify it as a work item?

[lucab]

Are we already doing that in FCOS? Should we identify it as a work item?

No, and yes. Specifically, we should go through https://github.com/coreos/init/tree/master/udev and port the relevant bits into a dedicated source (coreos-cloud-rules?), to be packaged as a rpm.

[dustymabe]

Thanks @lucab - I wonder what we do for Fedora/RHEL today. Do we just not ship those udev rules or are they shipped by some other package already?

[mike-nguyen]

I created #104 for the work needed from this card.

[dustymabe]

@mike-nguyen can you open a PR to the design doc similar to #102 ? and then we can close this ticket out

[mike-nguyen]

@dustymabe will do!

[dustymabe]

closing this since #108 was merged and #104 will address all gaps