Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ECSTasks module #988

Merged
merged 2 commits into from
May 22, 2023
Merged

Conversation

sstoops
Copy link
Contributor

@sstoops sstoops commented Apr 27, 2023

This PR includes a new module to handle ECS Tasks. While most ECS Tasks are handled through other dependent resources, there still exists an opportunity for an unhandled task to error out the AWS Nuke cleanup process. Details below.

Task Coverage

  • ✅ When a task is launched within an ECS Cluster containing EC2 container instances, it will be cleaned up when the EC2 instance is removed from the cluster. While just removing the instance does not stop the task itself, its assumed that the EC2 instance will be promptly shut down, thus, eliminating the task.
  • ✅ When a task is launched via an ECS Fargate Service, the task is removed by the cleanup of the ECS Service.
  • ❌ When a task is launched as a standalone ECS Fargate task, outside of a service, it remains unhandled by AWS Nuke. This new module resolves this scenario.

Testing

Tests were performed to cover both existing handled tasks and those that have been found to be unhandled.

First, a standalone Fargate task was launched within a new cluster and the existing AWS Nuke was used to clean these resources up. A failure was thrown with a message about the running task:

ERRO[0036] There are resources in failed state, but none are ready for deletion, anymore. 

us-east-1 - ECSCluster - arn:aws:ecs:us-east-1:<redacted>:cluster/ecs-cluster-1682378325 - failed
ERRO[0036] ClusterContainsTasksException: The Cluster cannot be deleted while Tasks are active. 
Error: failed

Then, using this new module, AWS Nuke was run again and it found the task and handled it as expected:

us-east-1 - ECSCluster - arn:aws:ecs:us-east-1:<redacted>:cluster/ecs-cluster-1682379196 - would remove
us-east-1 - ECSTask - arn:aws:ecs:us-east-1:<redacted>:task/ecs-cluster-1682379196/72e8dfd81f524b378034958b06d58ca8 -> arn:aws:ecs:us-east-1:<redacted>:cluster/ecs-cluster-1682379196 - [ClusterARN: "arn:aws:ecs:us-east-1:<redacted>:cluster/ecs-cluster-1682379196", TaskARN: "arn:aws:ecs:us-east-1:<redacted>:task/ecs-cluster-1682379196/72e8dfd81f524b378034958b06d58ca8"] - would remove

Setup code

The following code was used to launch a standalone Fargate task.

# Get default VPC ID
echo "getting default VPC ID"
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true --query 'Vpcs[0].VpcId' --output text)

# Get default subnets as comma separated string
echo "getting default subnets"
SUBNETS=$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC_ID --query 'Subnets[*].SubnetId' --output text | tr '\t' ',')

# Get default security group
echo "getting default security group"
SECURITY_GROUP=$(aws ec2 describe-security-groups --filters Name=vpc-id,Values=$VPC_ID --query 'SecurityGroups[0].GroupId' --output text)

# Create an ECS cluster 
echo "creating ECS cluster"
CLUSTER_NAME="ecs-cluster-$(date +%s)"
aws ecs create-cluster --cluster-name $CLUSTER_NAME
aws ecs list-clusters

echo "creating ECS task definition"
TASK_DEFINITION_NAME="ecs-task-$(date +%s)"
cat <<EOF > task-definition.json
{
  "family": "$TASK_DEFINITION_NAME",
  "networkMode": "awsvpc",
  "containerDefinitions": [
    {
      "name": "sleep",
      "image": "alpine",
      "cpu": 10,
      "memory": 128,
      "essential": true, 
      "command": [
          "sleep", "360"
      ]
    }
  ],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "cpu": "256", 
  "memory": "512"
}
EOF

aws ecs register-task-definition --cli-input-json file://task-definition.json
echo "listing ECS task definitions"
aws ecs list-task-definitions

# Run the task
echo "running ECS task"
aws ecs run-task --cluster $CLUSTER_NAME --task-definition $TASK_DEFINITION_NAME --launch-type FARGATE --network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$SECURITY_GROUP],assignPublicIp=ENABLED}"

echo "listing ECS tasks"
aws ecs list-tasks --cluster $CLUSTER_NAME

@sstoops sstoops requested a review from a team as a code owner April 27, 2023 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants