Skupper HTTP Demo

A demo show using Skupper with Kubernetes HTTP services to show seamless failover to next available cloud.

Pre-requisites

• minikube

• kubectl

• Java 8

• Nodejs

  kustomize

  Since we need to deploy to multiple clouds, using kustomize allows us to patch Kubernetes manifests that are unique for each cloud.

  skaffold

  Skaffold allows us to do seamless deployment into kube clusters. Highly useful for development use cases

  skupper

  Skupper is a layer 7 service interconnect. It enables secure communication across Kubernetes clusters with no VPNs or special firewall rules.

This Demo uses Google Translation, you need to have your Google API key (Json) which we will use it later when deploying lingua-greeter.

Setup

Important

Create three separate directories for earth,mars and jupiter which will hold the $KUBECONFIG for each environment. mkdir -p ./{earth,mars,jupiter}/.kube

Earth

pushd earth > /dev/null
export KUBECONFIG="$(pwd)/.kube/config"
minikube -p earth start \
  --service-cluster-ip-range='10.96.10.0/27'
popd > /dev/null

Enable Ingress

The minikube earth need to gave ingress enabled for accessing the services outside the cluster. Let us haproxy ingress

kubectl label node earth role=ingress-controller
kubectl create -f <a href="https://haproxy-ingress.github.io/resources/haproxy-ingress.yaml" class="bare">https://haproxy-ingress.github.io/resources/haproxy-ingress.yaml</a>

Watch the ingress-controller daemonset for the nginx ingress controller to be up and running:

kubectl get daemonset -n ingress-controller

In few minutes you should see the nginx ingress pod in the ingress-controller namespace:

NAME              DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR             AGE
haproxy-ingress   1         1         1       1            1           role=ingress-controller   90s

Once the daemonset is running in ingress-controller, check the nginx ingress controller pod to be up and running:

watch kubectl get pods -n ingress-controller

In few minutes you should see the nginx ingress pod in the ingress-controller namespace:

NAME                    READY   STATUS    RESTARTS   AGE
haproxy-ingress-f64ps   1/1     Running   0          30s

Mars

pushd mars > /dev/null
export CLOUD_PROFILE='mars'
export KUBECONFIG="$(pwd)/.kube/config"
minikube -p mars start \
  --service-cluster-ip-range='10.96.3.0/27'
popd > /dev/null

Tunnel

Minikube tunnel allows us to interact with Kubernetes services using its CLUSTER-IP from the host. Once minikube mars started, open a new terminal and run

pushd mars > /dev/null
export KUBECONFIG="$(pwd)/.kube/config"
minikube -p mars tunnel
popd > /dev/null

Jupiter

pushd jupiter > /dev/null
export CLOUD_PROFILE='jupiter'
export KUBECONFIG="$(pwd)/.kube/config"
minikube -p jupiter start \
  --service-cluster-ip-range='172.16.3.0/27'
popd > /dev/null

Tunnel

Minikube tunnel allows us to interact with Kubernetes services using its CLUSTER-IP from the host. Once minikube jupiter started, open a new terminal and run

pushd jupiter > /dev/null
export KUBECONFIG="$(pwd)/.kube/config"
minikube -p jupiter tunnel
popd > /dev/null

Skupper Setup

Mars(minikube)

skupper init --id  mars

Wait for skupper to initialize skupper status

Create connection token

skupper connection-token ../mars-token.yaml

Jupiter(minikube)

skupper init --id jupiter

Wait for skupper to initialize skupper status

skupper connection-token ../jupiter-token.yaml
skupper connect  --connection-name mars  ../mars-token.yaml

Earth(minikube)

Create and change to use the namespace demo

kubectl create ns demo
kubectl config set-context --current --namespace=demo

skupper init --edge --id earth

Wait for skupper to initialize skupper status

# lower the cost greater the affinity
skupper connect  --connection-name mars --cost 1 ../mars-token.yaml
# higher the cost lesser the affinity
skupper connect  --connection-name jupiter --cost 20 ../jupiter-token.yaml

Verify skupper setup

• skupper list-connectors

• skupper list-exposed

• skupper check-connection all

Deploy lingua-greeter

Expose Skupper Services

Jupiter

Ensure your context is in cloud jupiter by running the command ` kubectl config current-context`

cd $PROJECT_HOME/jupiter
export GOOGLE_APPLICATION_CREDENTIALS=<path to your Google API Key Json file>
kubectl create secret generic google-cloud-creds \
  --from-file="google-cloud-credentials.json=$GOOGLE_APPLICATION_CREDENTIALS"
skupper expose deployment lingua-greeter \
 --address lingua-greeter \
 --port 8080 --protocol tcp --target-port 8080

Mars

Ensure your context is in cloud mars by running the command ` kubectl config current-context`

cd $PROJECT_HOME/mars
export GOOGLE_APPLICATION_CREDENTIALS=<path to your Google API Key Json file>
kubectl create secret generic google-cloud-creds \
  --from-file="google-cloud-credentials.json=$GOOGLE_APPLICATION_CREDENTIALS"
skupper expose deployment lingua-greeter \
 --address lingua-greeter \
 --port 8080 --protocol tcp --target-port 8080

Deploy greetings-ticker

Since the greetings-ticker is a SPA, we need to have the external url for accessing the lingua-greeter service via HTML. Run the following command to expose lingua-greeter

export MINIKUBE_IP=$(minikube -p earth ip)
cat k8s/lingua-greeter-ingress.yaml| envsubst | kubectl apply -n demo -f -

Get the ingress URL:

LINGUA_GREETER_INGRESS_URL="$(kubectl get -n demo ingress lingua-greeter -o jsonpath='{.spec.rules[0].host}')"
echo $LINGUA_GREETER_INGRESS_URL

Check the health of the back end:

http $LINGUA_GREETER_INGRESS_URL/health/live

Ensure your context is in cloud mars by running the command ` kubectl config current-context`.

Verify if your are in demo namespace by running the command kubectl config view --minify | grep namespace:.

cd $PROJECT_HOME/earth
eval $(minikube -p earth docker-env)
# create .env.production
echo "NODE_ENV=production\nVUE_APP_MESSAGES_URL=http://$LINGUA_GREETER_INGRESS_URL" | tee .env.production

Before we deploy the greetings-ticker, check the list of services in the demo namespace, since we exposed lingua-greeter on other clouds (jupiter/mars) we should see it listed in earth, as we connected earth with jupiter and mars earlier.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE lingua-greeter ClusterIP 10.96.10.2 <none> 8080/TCP 10m skupper-messaging ClusterIP 10.96.10.11 <none> 5671/TCP 22m

Deploy the application using the command:

skaffold run --tail

Open the application in the browser using minikube -p earth -n demo service greetings-ticker.

Verify load balancing

You should by default start to see the messages coming from mars, after few messages go to mars terminal and scale the lingua-greeter service to 0 and now you will see the responses from Jupiter only and do the vice versa to see the responses flipping back mars.

Important

Since we have set the cost to mars, the application will always try to go to mars by default