Indexify-aws-deployment

This document provides a comprehensive guide for setting up and configuring an eks infrastructure running using AWS, Terraform, and Kubernetes tools.

1. Install AWS + Terraform CLI

Install terraform cli docs

Install aws cli docs

Install eksctl docs

2. Create AWS Credentials

• In the console navigate to IAM.
• Create a new user, for our example we will name this user indexify-user (this can be named anything), where we will create aws credentials on behalf of.
• Go to indexify-user Security Credentials tab and create access key for CLI
• configure aws cli with your credentials with

  aws configure

3. Setup IAM Policy + Role

• Create a new policy named TerraformPolicy with the following.

  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "Statement1",
        "Effect": "Allow",
        "Action": [
          "ec2:*",
          "rds:*",
          "iam:*",
          "logs:*",
          "kms:*",
          "eks:*",
          "s3:*",
          "cloudformation:*"
        ],
        "Resource": [
          "*"
        ]
      }
    ]
  }

  *Note this policy is just an example, you may want to change this policy based on your needs.
• Add TerraformPolicy to indexify-user permissions.

4. Initialize terraform and apply resources

First you want to change values marked changeme inside of terraform/variables.tf. This is for your database password and s3 bucket name

cd terraform
terraform init
terraform apply

5. Configure Kubernetes files

Once all of your resources have been created with terraform it is time to setup your kubeconfig. Make sure you have installed kubectl

Update the values inside of k8/indexify-configmap.yml marked changeme, this includes

• database_url
• your s3 bucket name
• pgvector database_url (same as above)

Update the environment variables AWS Access Keys inside of indexify-server.yml and indexify-minilm-l6-extractor.yml.

Set your kubeconfig to the cluster

aws eks update-kubeconfig --region us-east-1 --name indexify-cluster

6. k8 Load Balancer Setup

Our k8 configuration will provision a load balancer for us, we need to do a few steps in order for this to work.

Create AWSLoadBalancerControllerIAMPolicy Policy

download the policy

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json

create the load balancer controller iam policy

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam_policy.json

Create Service Account

this command uses eksctl

create service account, update this command to use your aws account number

eksctl create iamserviceaccount \
  --cluster=indexify-cluster \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::YOURAWSACCOUNT:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve

verify role was created

aws iam get-role --role-name AmazonEKSLoadBalancerControllerRole --query Role.AssumeRolePolicyDocument

verify policy that you attached is attached to the role

aws iam list-attached-role-policies --role-name AmazonEKSLoadBalancerControllerRole --query AttachedPolicies --output text

output should be something like this arn:aws:iam::ACCOUNTNAME:policy/AWSLoadBalancerControllerIAMPolicy

Set a variable to store the Amazon Resource Name (ARN) of the policy that you want to use. Replace my-policy with the name of the policy that you want to confirm permissions for.

*Specify the output of the previous command

export policy_arn=PREVIOUS_COMMAND_OUTPUT

view default version of policy

aws iam get-policy --policy-arn $policy_arn

confirm k8 service account is annotated with the role

kubectl describe serviceaccount aws-load-balancer-controller -n kube-system

Create OICD provider

determine OICD issuer ID for our cluster

oidc_id=$(aws eks describe-cluster --name indexify-cluster --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)

echo $oidc_id

determined if we already have oidc provider with cluster issuer id on our account. if output is returned skip next step

aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4

the following command will create oidc provider

eksctl utils associate-iam-oidc-provider --region=us-east-1 --cluster=indexify-cluster --approve

Configure Service account to assume an IAM role

Any pods that are configured to use this service account can access any aws service the role has permission to access.

Add help repo

helm repo add eks https://aws.github.io/eks-charts
helm repo update eks

Install AWS Load Balancer

kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller/crds?ref=master"

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=indexify-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

Verify installation

kubectl get deployment -n kube-system aws-load-balancer-controller

Apply ingress

kubectl apply -f k8/ingress.yml

check logs of aws load balancer controller to make sure no errors occured

kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller

7. Apply the rest of the k8 configuration

apply k8

kubectl apply -f k8/namespace.yml
kubectl apply -f k8/

8. Tests

The load balancer is provisioned with k8, go to your load balancers on the aws console and here you can find your dns name

To make these tests easier let's set up an env variable with your dns name

export ALB_URL=....

Add some data

curl -v -X POST ${ALB_URL}/repositories/default/add_texts \
-H "Content-Type: application/json" \
-d '{"documents": [ 
        {"text": "Indexify is amazing!"},
        {"text": "Indexify is a retrieval service for LLM agents!"}, 
        {"text": "Kevin Durant is the best basketball player in the world."}
    ]}'

Bind our minilm extractor

curl -v -X POST ${ALB_URL}/repositories/default/extractor_bindings \
-H "Content-Type: application/json" \
-d '{
        "extractor": "tensorlake/minilm-l6-extractor",
        "name": "minil6"
    }'

Get indexes

curl ${ALB_URL}/repositories/default/indexes

Perform a search

curl -v -X POST ${ALB_URL}/repositories/default/search \
-H "Content-Type: application/json" \
-d '{
        "index": "minil6.embedding",
        "query": "basketball", 
        "k": 3
    }'