elder-peeler

Simple "Kubernetes Event Driven Autoscaling" KEDA demonstration for AWS EKS.

This repository:

1. Deploys a simple SQS consumer (and shares SQS values w/prometheus).
2. Deploys KEDA to cluster, to scale the consumer depending upon SQS message population.

SQS Consumer Application

1. Assumes same AWS account, region, etc.
2. Create a AWS SQS queue
3. Create a IAM role
   1. example
   2. Be sure to add SQS permissions
4. Create a namespace for your test
   1. kubectl create ns guytest
5. Create service account
   1. Be sure to update the annotation to reflect the IAM role.
   2. kubectl apply -f service_account.yaml
   3. Create an IRSA account that grants access to SQS
      1. iam_role.json is an example
      2. be sure to add SQS access to role
6. Deploy the SQS consumer to your cluster.
   1. "elder-peeler" is the name of the SQS consumer.
   2. There is a Dockerfile to help you get started.
      1. SQS queue name is defined within Dockerfile
   3. There is a deployment.yaml to help you get started
7. Sanity test.
   1. There should be an elder-peeler pod active in your namespace.
   2. Write some messages to SQS (via the console or try producer.py
   3. Tail the pod log and it should be writing happy messages about message consumption.
8. Optional prometheus step
   1. Apply pod-monitor.yaml and elder-peeler and prometheus will scrape elder-peeler
      1. exposes peeler_messages_available and peeler_messages_consumed

Apply KEDA

1. Create a IAM role
   1. example
   2. Be sure to add SQS permissions
2. Install KEDA via helm
   1. helm install keda kedacore/keda --namespace keda --create-namespace --version 2.13.2 -f values.yaml
   2. Note that values.yaml contains the ARN of keda role.
   3. It takes a minute for gRPC to get connected
3. Apply ScaledObject
   1. kubectl apply -f scaled.yaml
4. Verify ScaledObject
   1. kubectl describe scaledobject guyscaler -n guytest
   2. my results
5. Test KEDA
   1. Write messages to your SQS queue.
   2. Before KEDA, there would have been only a single replica for all traffic but now the scaler should create many pods handle the increase in SQS traffic.
   3. After the SQS queue is emptied, the pods will all be removed by KEDA
6. Cleanup
   1. Be sure to read cleanup instructions on KEDA website, it isn't enough to invoke helm delete