This document provides a comprehensive guide for setting up and configuring an eks infrastructure running using AWS, Terraform, and Kubernetes tools.
- In the console navigate to IAM.
- Create a new user, for our example we will name this user indexify-user (this can be named anything), where we will create aws credentials on behalf of.
- Go to indexify-user
Security Credentials
tab and create access key for CLI - configure aws cli with your credentials with
aws configure
- Create a new policy named
TerraformPolicy
with the following.*Note this policy is just an example, you may want to change this policy based on your needs.{ "Version": "2012-10-17", "Statement": [ { "Sid": "Statement1", "Effect": "Allow", "Action": [ "ec2:*", "rds:*", "iam:*", "logs:*", "kms:*", "eks:*", "s3:*", "cloudformation:*" ], "Resource": [ "*" ] } ] }
- Add
TerraformPolicy
toindexify-user
permissions.
First you want to change values marked changeme inside of terraform/variables.tf. This is for your database password and s3 bucket name
cd terraform
terraform init
terraform apply
Once all of your resources have been created with terraform it is time to setup your kubeconfig. Make sure you have installed kubectl
Update the values inside of k8/indexify-configmap.yml
marked changeme, this includes
- database_url
- your s3 bucket name
- pgvector database_url (same as above)
Update the environment variables AWS Access Keys inside of indexify-server.yml
and indexify-minilm-l6-extractor.yml
.
Set your kubeconfig to the cluster
aws eks update-kubeconfig --region us-east-1 --name indexify-cluster
Our k8 configuration will provision a load balancer for us, we need to do a few steps in order for this to work.
download the policy
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json
create the load balancer controller iam policy
aws iam create-policy \
--policy-name AWSLoadBalancerControllerIAMPolicy \
--policy-document file://iam_policy.json
this command uses eksctl
create service account, update this command to use your aws account number
eksctl create iamserviceaccount \
--cluster=indexify-cluster \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::YOURAWSACCOUNT:policy/AWSLoadBalancerControllerIAMPolicy \
--approve
verify role was created
aws iam get-role --role-name AmazonEKSLoadBalancerControllerRole --query Role.AssumeRolePolicyDocument
verify policy that you attached is attached to the role
aws iam list-attached-role-policies --role-name AmazonEKSLoadBalancerControllerRole --query AttachedPolicies --output text
output should be something like this arn:aws:iam::ACCOUNTNAME:policy/AWSLoadBalancerControllerIAMPolicy
Set a variable to store the Amazon Resource Name (ARN) of the policy that you want to use. Replace my-policy with the name of the policy that you want to confirm permissions for.
*Specify the output of the previous command
export policy_arn=PREVIOUS_COMMAND_OUTPUT
view default version of policy
aws iam get-policy --policy-arn $policy_arn
confirm k8 service account is annotated with the role
kubectl describe serviceaccount aws-load-balancer-controller -n kube-system
determine OICD issuer ID for our cluster
oidc_id=$(aws eks describe-cluster --name indexify-cluster --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)
echo $oidc_id
determined if we already have oidc provider with cluster issuer id on our account. if output is returned skip next step
aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4
the following command will create oidc provider
eksctl utils associate-iam-oidc-provider --region=us-east-1 --cluster=indexify-cluster --approve
Any pods that are configured to use this service account can access any aws service the role has permission to access.
helm repo add eks https://aws.github.io/eks-charts
helm repo update eks
kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller/crds?ref=master"
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=indexify-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller
kubectl get deployment -n kube-system aws-load-balancer-controller
kubectl apply -f k8/ingress.yml
check logs of aws load balancer controller to make sure no errors occured
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
apply k8
kubectl apply -f k8/namespace.yml
kubectl apply -f k8/
The load balancer is provisioned with k8, go to your load balancers on the aws console and here you can find your dns name
To make these tests easier let's set up an env variable with your dns name
export ALB_URL=....
curl -v -X POST ${ALB_URL}/repositories/default/add_texts \
-H "Content-Type: application/json" \
-d '{"documents": [
{"text": "Indexify is amazing!"},
{"text": "Indexify is a retrieval service for LLM agents!"},
{"text": "Kevin Durant is the best basketball player in the world."}
]}'
curl -v -X POST ${ALB_URL}/repositories/default/extractor_bindings \
-H "Content-Type: application/json" \
-d '{
"extractor": "tensorlake/minilm-l6-extractor",
"name": "minil6"
}'
curl ${ALB_URL}/repositories/default/indexes
curl -v -X POST ${ALB_URL}/repositories/default/search \
-H "Content-Type: application/json" \
-d '{
"index": "minil6.embedding",
"query": "basketball",
"k": 3
}'