From cc687dcf6cb586e32e63235164db5437b976df1c Mon Sep 17 00:00:00 2001 From: Samuel Karp Date: Wed, 19 Aug 2020 15:36:45 -0700 Subject: [PATCH] docs: add documentation for aws-ecs-1 variant --- BUILDING.md | 11 ++- QUICKSTART-ECS.md | 193 ++++++++++++++++++++++++++++++++++++++++++++++ README.md | 46 +++++++++-- 3 files changed, 241 insertions(+), 9 deletions(-) create mode 100644 QUICKSTART-ECS.md diff --git a/BUILDING.md b/BUILDING.md index e475162cc74..7a159e41bad 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -1,7 +1,7 @@ # Building Bottlerocket If you'd like to build your own image instead of relying on an Amazon-provided image, follow these steps. -You can skip to the [setup guide](QUICKSTART-EKS.md) to use an existing image in Amazon EC2. +You can skip to the [setup guide for Kubernetes](QUICKSTART-EKS.md) or the [setup guide for Amazon ECS](QUICKSTART-ECS.md) to use an existing image in Amazon EC2. (We're still working on other use cases!) ## Build an image @@ -40,8 +40,15 @@ To build an image, run: cargo make ``` +This will build an image for the default variant, `aws-k8s-1.17`. All packages will be built in turn, and then compiled into an `img` file in the `build/` directory. +To build an image for a different variant, run: + +``` +cargo make -e BUILDSYS_VARIANT=my-variant-here +``` + ### Register an AMI To use the image in Amazon EC2, we need to register the image as an AMI. @@ -74,4 +81,4 @@ Your new AMI ID will be printed at the end. ## Use your image -See the [setup guide](QUICKSTART-EKS.md) for information on running Bottlerocket images. +See the [setup guide for Kubernetes](QUICKSTART-EKS.md) or the [setup guide for Amazon ECS](QUICKSTART-ECS.md) for information on running Bottlerocket images. diff --git a/QUICKSTART-ECS.md b/QUICKSTART-ECS.md new file mode 100644 index 00000000000..1863574ea4b --- /dev/null +++ b/QUICKSTART-ECS.md @@ -0,0 +1,193 @@ +# Using a Bottlerocket AMI with Amazon ECS + +> The [ECS variant](./aws-ecs.md) of Bottlerocket is currently in a developer preview phase and we're looking for your +> [feedback](https://github.com/bottlerocket-os/bottlerocket#contact-us). +> We'd love for you to try it out! + +[Amazon Elastic Container Service (Amazon ECS)](https://ecs.aws) is a highly scalable, fast container management service that makes it easy to run, stop, and manage containers on a cluster. +Your containers are defined in a task definition which you use to run individual tasks or as a service. + +This quickstart will walk through setting up an Amazon ECS cluster with Bottlerocket container instances (using the EC2 launch type). +Check out the [Amazon ECS developer guide](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for an overview of ECS. + +## Prerequisites + +Before you begin, be sure that you've completed the steps in +[Setting up with Amazon ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html) +and that your AWS user has either the [`AdministratorAccess`](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/AdministratorAccess) policy +or the permissions specified in the [Amazon ECS First Run Wizard Permissions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security_iam_id-based-policy-examples.html#first-run-permissions) IAM policy example. + +You'll also need [aws-cli](https://aws.amazon.com/cli/) set up to interact with AWS. + + +## Create a cluster + +An Amazon ECS cluster is a logical grouping of tasks, services, and container instances. +For more information about clusters, see +[Amazon ECS clusters](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html). + +You can create a cluster with the AWS CLI as follows: + +``` +aws ecs --region us-west-2 create-cluster --cluster-name bottlerocket +``` + +> Note: The command above and subsequent examples include the AWS region, so change it from `us-west-2` if you operate in another region. + +## Finding an AMI + +You can either use an official AMI provided by Amazon or build an AMI yourself using +[our guide](https://github.com/bottlerocket-os/bottlerocket/blob/develop/BUILDING.md). + +The official AMI IDs are stored in +[public SSM parameters](https://docs.aws.amazon.com/systems-manager/latest/userguide/parameter-store-public-parameters.html). +To find the `aws-ecs-1` variant for the `x86_64` architecture in the `us-west-2` region, you can use the following command: + +``` +aws ssm get-parameter --region us-west-2 --name "/aws/service/bottlerocket/aws-ecs-1/x86_64/latest/image_id" --query Parameter.Value --output text +``` + +If you have `jq` and would like a bit more information, try this: + +``` +aws ssm get-parameters --region us-west-2 \ + --names "/aws/service/bottlerocket/aws-ecs-1/x86_64/latest/image_id" \ + "/aws/service/bottlerocket/aws-ecs-1/x86_64/latest/image_version" \ + --output json | jq -r '.Parameters | .[] | "\(.Name): \(.Value) (updated \(.LastModifiedDate | gmtime | strftime("%c")) UTC)"' +``` + +You can replace the architecture (`x86_64`) and region (`us-west-2`) to look for other images. +If you know a specific Bottlerocket version you'd like to use, you can replace `latest` with that version. + +You can also see all available parameters +[through the web console](https://us-west-2.console.aws.amazon.com/systems-manager/parameters/#list_parameter_filters=Path:Recursive:%2Faws%2Fservice%2Fbottlerocket). + +## Launching your first instance + +In order to launch a Bottlerocket instance into your ECS cluster, you'll first need some information about the resources in your AWS account. + +### Subnet info + +You should either have a default virtual private cloud (VPC) or have already +[created a VPC](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html#create-a-vpc) +in your account. + +To find your default VPC, run this command. +(If you use an AWS region other than "us-west-2", make sure to change that.) + +``` +aws ec2 describe-vpcs \ + --region us-west-2 \ + --filters=Name=isDefault,Values=true \ + | jq --raw-output '.Vpcs[].VpcId' +``` + +If you want to use a different VPC you created, run this to get the ID for your VPC. +Make sure to change VPC_NAME to the name of the VPC you created. +(If you use an EC2 region other than "us-west-2", make sure to change that too.) + +``` +aws ec2 describe-vpcs \ + --region us-west-2 \ + --filters=Name=tag:Name,Values=VPC_NAME \ + | jq --raw-output '.Vpcs[].VpcId' +``` + +Next, run this to get information about the subnets in your VPC. +It will give you a list of the subnets and tell you whether each is public or private. +Make sure to change VPC_ID to the value you received from the previous command. +(If you use an EC2 region other than "us-west-2", make sure to change that too.) + +``` +aws ec2 describe-subnets \ + --region us-west-2 \ + --filter=Name=vpc-id,Values=VPC_ID \ + | jq '.Subnets[] | {id: .SubnetId, public: .MapPublicIpOnLaunch, az: .AvailabilityZone}' +``` + +You'll want to pick one and save it for the launch command later. + +You can choose whether you want public or private. +* Choose private for production deployments to get maximum isolation of instances. +* Choose public to more easily debug your instance. + These subnets have an Internet Gateway, so if you add a public IP address to your instance, you can talk to it. + (You can manually add an Internet Gateway to a private subnet later, so this is a reversible decision.) + +Note that if you choose to use the public subnet, you'll need your instance to have a publicly accessible IP address. +That either means adding `--associate-public-ip-address` to the launch command below, or attaching an Elastic IP address after launch. +There will be a reminder about this when we talk about the launch command. + +Finally, note that if you want to launch in a specific availability zone, make sure you pick a subnet that matches; the AZ is listed right below the public/private status. + +### IAM role + +The instance we launch needs to be associated with an IAM role that allows for communication with ECS. + +ECS provides a +[managed policy](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_managed_policies.html#AmazonEC2ContainerServiceforEC2Role) +with all of the appropriate permissions. +If you've used ECS before, you may already have an appropriate role in your account called `ecsInstanceRole`. +If you do not, you can +[follow the instructions in the ECS Developer Guide to create a role](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/instance_IAM_role.html). + +Note down the instance role name in your account for the instructions below. + +#### Enabling SSM + +If you add SSM permissions, you can use Bottlerocket's default SSM agent to get a shell session on the instance. + +To attach the role policy for SSM permissions, run the following (replacing INSTANCE_ROLE_NAME with the name of your instance role): + +``` +aws iam attach-role-policy \ + --role-name INSTANCE_ROLE_NAME \ + --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore +``` + +Next, to retrieve the instance profile name used to launch instances, run this: + +``` +aws iam list-instance-profiles-for-role --role-name INSTANCE_ROLE_NAME --query "InstanceProfiles[*].InstanceProfileName" --output text +``` + +Note this down as the INSTANCE_PROFILE_NAME for the final launch command. + +### Connecting to your cluster + +For the instance to be able to communicate with ECS, we need to make sure to configure the instance with the name of the cluster. + +Create a file called `userdata.toml` with the following contents, where CLUSTER_NAME is the name of the cluster you created above (for example, "bottlerocket"). + +``` +[settings.ecs] +cluster = "CLUSTER_NAME" +``` + +If you want to customize the behavior of your instance further, you can find the full set of supported settings [here](README.md#settings). + +### Launch! + +Now we can launch a Bottlerocket instance in our cluster! + +There are a few values to make sure you change in this command: +* YOUR_KEY_NAME: your SSH keypair name, as registered with EC2 +* SUBNET_ID: the subnet you selected earlier + * If you chose a public subnet, either add `--associate-public-ip-address` to the command, or attach an Elastic IP afterward. +* BOTTLEROCKET_AMI_ID: the Amazon-provided AMI ID you found above, or the ID of an AMI you registered +* userdata.toml: the path to the user data file you created earlier +* INSTANCE_PROFILE_NAME: the IAM instance profile you created, e.g. `ecsInstanceRole` + +``` +aws ec2 run-instances --key-name YOUR_KEY_NAME \ + --subnet-id SUBNET_ID \ + --image-id BOTTLEROCKET_AMI_ID \ + --instance-type c3.large \ + --region us-west-2 \ + --tag-specifications 'ResourceType=instance,Tags=[{Key=bottlerocket,Value=quickstart}]' \ + --user-data file://userdata.toml \ + --iam-instance-profile Name=INSTANCE_PROFILE_NAME +``` + +And remember, if you used a public subnet, add `--associate-public-ip-address` or attach an Elastic IP after launch. + +Once it launches, you should be able to run tasks on your Bottlerocket instance using the ECS API and console. diff --git a/README.md b/README.md index bab10b2ec35..05cdcbcfa45 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ Welcome to Bottlerocket! Bottlerocket is a free and open-source Linux-based operating system meant for hosting containers. Bottlerocket is currently in a developer preview phase and we’re looking for your [feedback](#contact-us). -If you’re ready to jump right in, read our [QUICKSTART](QUICKSTART-EKS.md) to try Bottlerocket in an Amazon EKS cluster. +If you’re ready to jump right in, read our [QUICKSTART for Kubernetes](QUICKSTART-EKS.md) to try Bottlerocket in an Amazon EKS cluster or our [QUICKSTART for Amazon ECS](QUICKSTART-ECS.md) to try Bottlerocket in an Amazon ECS cluster. Bottlerocket focuses on security and maintainability, providing a reliable, consistent, and safe platform for container-based workloads. This is a reflection of what we've learned building operating systems and services at Amazon. @@ -39,7 +39,7 @@ You can let us know about things that seem difficult, or even ways you might lik ## Variants -To start, we're focusing on use of Bottlerocket as a host OS in AWS EKS Kubernetes clusters. +To start, we're focusing on use of Bottlerocket as a host OS in AWS EKS Kubernetes clusters and Amazon ECS clusters. We’re excited to get early feedback and to continue working on more use cases! Bottlerocket is architected such that different cloud environments and container orchestrators can be supported in the future. @@ -47,7 +47,8 @@ A build of Bottlerocket that supports different features or integration characte The artifacts of a build will include the architecture and variant name. For example, an `x86_64` build of the `aws-k8s-1.17` variant will produce an image named `bottlerocket-aws-k8s-1.17-x86_64--.img`. -Our first supported variants, `aws-k8s-1.15`, `aws-k8s-1.16`, and `aws-k8s-1.17`, supports EKS as described above. +Our first supported variants, `aws-k8s-1.15`, `aws-k8s-1.16`, and `aws-k8s-1.17`, support EKS as described above. +We also have a new `aws-ecs-1` variant designed to work with ECS. ## Architectures @@ -62,9 +63,11 @@ It describes: * how to build an image * how to register an EC2 AMI from an image -To get started using Bottlerocket, please see [QUICKSTART](QUICKSTART-EKS.md). -It describes: -* how to set up a Kubernetes cluster, so your Bottlerocket instance can run pods +Bottlerocket is best used with a container orchestrator. +To get started with Kubernetes, please see [QUICKSTART-EKS](QUICKSTART-EKS.md). +To get started with Amazon ECS, please see [QUICKSTART-ECS](QUICKSTART-ECS.md). +These guides describe: +* how to set up a cluster with the orchestrator, so your Bottlerocket instance can run containers * how to launch a Bottlerocket instance in EC2 ## Exploration @@ -331,9 +334,10 @@ In this format, "settings.kubernetes.cluster-name" refers to the same key as in #### Kubernetes settings +See the [setup guide](QUICKSTART-EKS.md) for much more detail on setting up Bottlerocket and Kubernetes. + The following settings must be specified in order to join a Kubernetes cluster. You should [specify them in user data](#using-user-data). -See the [setup guide](QUICKSTART-EKS.md) for *much* more detail on setting up Bottlerocket and Kubernetes. * `settings.kubernetes.cluster-name`: The cluster name you chose during setup; the [setup guide](QUICKSTART-EKS.md) uses "bottlerocket". * `settings.kubernetes.cluster-certificate`: This is the base64-encoded certificate authority of the cluster. * `settings.kubernetes.api-server`: This is the cluster's Kubernetes API endpoint. @@ -360,6 +364,33 @@ The following settings are set for you automatically by [pluto](sources/api/) ba * `settings.kubernetes.node-ip`: The IPv4 address of this node. * `settings.kubernetes.pod-infra-container-image`: The URI of the "pause" container. +#### Amazon ECS settings + +See the [setup guide](QUICKSTART-ECS.md) for much more detail on setting up Bottlerocket and ECS. + +The following settings are optional and allow you to configure how your instance joins an ECS cluster. +Since joining a cluster happens at startup, they need to be [specified in user data](#using-user-data). +* `settings.ecs.cluster`: The name or [ARN](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) of your Amazon ECS cluster. + If left unspecified, Bottlerocket will join your `default` cluster. +* `settings.ecs.instance-attributes`: [Attributes](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-placement-constraints.html#attributes) in the form of key, value pairs added when registering the container instance in the cluster. + * Example user data for setting up attributes: + ``` + [settings.ecs.instance-attributes] + attribute1 = "foo" + attribute2 = "bar" + ``` + +The following settings are optional and allow you to further configure your cluster. +These settings can be changed at any time. +* `settings.ecs.logging-drivers`: The list of logging drivers available on the container instance. + The ECS agent running on a container instance must register available logging drivers before tasks that use those drivers are eligible to be placed on the instance. + Bottlerocket enables the `json-file`, `awslogs`, and `none` drivers by default. +* `settings.ecs.allow-privileged-containers`: Whether launching privileged containers is allowed on the container instance. + If this value is set to false, privileged containers are not permitted. + Bottlerocket sets this value to false by default. +* `settings.ecs.loglevel`: The level of verbosity for the ECS agent's logs. + Supported values are `debug`, `info`, `warn`, `error`, and `crit`, and the default is `info`. + #### Updates settings * `settings.updates.metadata-base-url`: The common portion of all URIs used to download update metadata. @@ -482,6 +513,7 @@ We currently package the following major third-party components: * containerd ([background](https://containerd.io/), [packaging](packages/containerd/)) * Kubernetes ([background](https://kubernetes.io/), [packaging](packages/kubernetes-1.15/)) * aws-iam-authenticator ([background](https://github.com/kubernetes-sigs/aws-iam-authenticator), [packaging](packages/aws-iam-authenticator/)) +* Amazon ECS agent ([background](https://github.com/aws/amazon-ecs-agent), [packaging](packages/ecs-agent/)) For further documentation or to see the rest of the packages, see the [packaging directory](packages/).