Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds SSM capability to Packer #526

Merged
merged 1 commit into from
Jan 21, 2021
Merged

Adds SSM capability to Packer #526

merged 1 commit into from
Jan 21, 2021

Conversation

Nirvikalpa108
Copy link
Contributor

@Nirvikalpa108 Nirvikalpa108 commented Jan 20, 2021

What is the problem?

Security HQ was showing an error when trying to collect the security groups of the Deploy Tools account. After debugging, we learnt that AWS Trusted Advisor was unable to return a response for an account with over 200 security groups. The Deploy Tools account had over 850! Most of these were temporary Packer security groups created by Amigo. Whilst we wrote a script to delete the unused security groups, we wanted to find a way to stop Amigo creating all of these security groups.

What is the solution?

We decided to add SSM support to Packer and create two dedicated security groups (one for CODE and another for PROD), so that the security groups do not need any inbound access rules and also so that we no longer create a new temporary security group for each new Packer instance created by Amigo.

Support for the AWS Systems Manager session manager lets users manage EC2 instances without the need to open inbound ports, or maintain bastion hosts...Once the tunnel has been created all SSH communication will be tunnelled through SSM to the remote instance. https://www.packer.io/docs/builders/amazon/ebs#ssh_interface-1

How to test

We've tested this on CODE.

When deploying to PROD:

  1. upload the new cloudformation to amigo PROD
  2. copy the security group id that is created
  3. add this to the config in dynamodb
  4. push the latest change to teamcity and deploy

How can we measure success?

No more temporary Packer security groups are created by Amigo, we just have 2 dedicated security groups.

@Nirvikalpa108 Nirvikalpa108 force-pushed the packer-ssm branch 2 times, most recently from 990382b to 0c22950 Compare January 20, 2021 12:29
@Nirvikalpa108 Nirvikalpa108 marked this pull request as ready for review January 20, 2021 12:39
@Nirvikalpa108 Nirvikalpa108 force-pushed the packer-ssm branch 2 times, most recently from 83a4317 to c4deefb Compare January 20, 2021 13:12
@@ -267,6 +274,9 @@ Resources:
unzip -d /opt/packer /tmp/packer_*_linux_amd64.zip
echo 'export PATH=${!PATH}:/opt/packer' > /etc/profile.d/packer.sh

wget -P /tmp https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb
dpkg -i /tmp/session-manager-plugin.deb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in a follow up PR, we should move both this step and the preceeding step to install packer into the AMI so that it doesn't run on every boot, but I think we can keep it here for now as it's a significant security improvement

@Nirvikalpa108 Nirvikalpa108 merged commit 4564ba9 into main Jan 21, 2021
@Nirvikalpa108 Nirvikalpa108 deleted the packer-ssm branch January 21, 2021 09:41
@Nirvikalpa108
Copy link
Contributor Author

This change is now on PROD and working 👍🏽

Comment on lines +115 to +118
- Effect: Allow
Action:
- iam:GetInstanceProfile
Resource: !Ref PackerInstanceProfile
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm looking at the policies we define as part of the CDK migration (see #598). I can't quite work out why this policy is needed (other than tracking it down to this PR). Are there any links available that explain it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we needed to add this so that the amigo instances are able to initial SSM sessions with the packer instances.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants