-
Notifications
You must be signed in to change notification settings - Fork 12
Lab 4 (Thu): Improving Secret Handling
We've observed in class that the environments of ECS Tasks are easily inspected from the ECS console. This exposes any secrets stored in those environments, including our AWS keys and Slack tokens.
In this lab, we will remove the AWS keys from the environment. Instead, we will use an AWS feature called "IAM Roles". ECS has a mechanism for assigning an ECS Task (our Slackbot in our case) to a role, granting that task all the privileges assigned to that role.
We have prepared a patch that implements this idea: it launches tasks into an IAM Role called LimboTaskRole and it stops passing AWS keys into the ECS Limbo Tasks. To
In Lab 1, you should've created a remote repository called upstream pointing to the tim77code/limbo repository. You can verify this remote exists with git remote:
bash$ git remote -v
origin git@github-rstata-verticloud:rstata-verticloud/limbo.git (fetch)
origin git@github-rstata-verticloud:rstata-verticloud/limbo.git (push)
upstream git@github.com:tim77code/limbo.git (fetch)
upstream git@github.com:tim77code/limbo.git (push)
bash$
If you missed this step, you can create this remote reference now by typing:
git remote add upstream https://github.com/tim77code/limbo.git
Bring your local repo up-to-date with git fetch:
bash$ git fetch --all --prune
Fetching origin
Fetching upstream
remote: Counting objects: 29, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 29 (delta 15), reused 26 (delta 15), pack-reused 0
Unpacking objects: 100% (29/29), done.
From github.com:tim77code/limbo
* [new branch] issue22-for-lab4 -> upstream/issue22-for-lab4
* [new branch] issue22-take3-master -> upstream/issue22-take3-master
4a2e422..7e7eeae master -> upstream/master
bash$
Create and checkout a new branch off of your master branch:
bash$ git checkout -b no-aws-tokens-master master
Switched to a new branch 'no-aws-tokens-master'
bash$
Pull our changes onto this new branch:
bash$ git pull upstream issue22-for-lab4
From github.com:tim77code/limbo
* branch issue22-for-lab4 -> FETCH_HEAD
Updating 4a2e422..d1a5de4
Fast-forward
bin/deploy.sh | 8 ++++++--
docker-compose.yml | 2 --
2 files changed, 6 insertions(+), 4 deletions(-)
bash$
Take a look at the changes we've just pulled in by typing
git diff master
First, take a look at differences to bin/deploy.sh:
diff --git a/bin/deploy.sh b/bin/deploy.sh
index 1ce5054..62f8481 100755
--- a/bin/deploy.sh
+++ b/bin/deploy.sh
@@ -58,7 +58,9 @@ case "$1" in
bin/ecr_push.sh
docker-compose --file cmds.yml run \
ecs-cli compose --file docker-compose.yml --region us-east-1 --cluster limbo \
- --project-name $SERVICE_NAME-$TYPE service up
+ --project-name $SERVICE_NAME-$TYPE \
+ --task-role-arn arn:aws:iam::560921689673:role/LimboTaskRole \
+ service up
;;
stop)
@@ -73,7 +75,9 @@ case "$1" in
bin/ecr_push.sh
docker-compose --file cmds.yml run \
ecs-cli compose --file docker-compose.yml --region us-east-1 --cluster limbo \
- --project-name $SERVICE_NAME-$TYPE service up
+ --project-name $SERVICE_NAME-$TYPE \
+ --task-role-arn arn:aws:iam::560921689673:role/LimboTaskRole \
+ service up
else
echo "Service not running, so not pushing an update."
fi
There is a lot of context in the diff output, but the main change is the addition of the following command-line option to our invocation of ecs-cli compose:
--task-role-arn arn:aws:iam::560921689673:role/LimboTaskRole
This command-line option tells AWS to assign the LimboTaskRole to our Slackbot's ECS Task when starting the task. Amazon make this very easy to do!
Now let's look at the diff of our docker-compose.yml file:
diff --git a/docker-compose.yml b/docker-compose.yml
index 12cbf25..1e9e754 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -3,8 +3,6 @@ version: "2"
services:
limbo:
environment:
- AWS_ACCESS_KEY_ID:
- AWS_SECRET_ACCESS_KEY:
LIMBO_NEEDMENTION:
LIMBO_CLOUDWATCH:
SLACK_TOKEN:
What you see is that we've dropped the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables from the environment of the limbo service. When our Limbo ECS tasks are started, they will no longer be given these environment variables. And these variables won't be needed, because the permissions granted by this access key is now granted instead through the LimboTaskRole.
Let's push this change to GitHub, so it can be built and deployed by Travis. Before doing so, let's make sure our Slackbot is up and running:
make ecs_start
(This will probably deploy an old version of your Slackbot, but that's okay: when you push our changes to GitHub, the changes will be deployed by Travis.)
Now let's push the changes to GitHub, so they can be built and deployed by Travis:
git push origin no-aws-tokens-master
Check to see that Travis builds and deploys this change and that your Slackbot is running as expected. Then, go into the ECS Web console, go into the details of your Task's container and confirm that the AWS_* environment variables have no longer been exported into your container.
(If you want to test this change locally by doing make ecs_start from your laptop, rather than using TravisCI to deploy the change, be sure to run make docker_build -- and make docker_test for good measure -- before running make ecs_start.)
We've eliminated the AWS secret from our ECS Task environments, but the Slack tokens remain. We've filed this additional step as GitHub Issue #43.