02.5-haproxy.md

Setting up an HA proxy for the API access

In order to support fault tolerance for the Kubernetes API as well as the access to the pods and services running on the ICC, a HAProxy is required. In the original documentation this capability was provided by the Google IaaS.

Install haproxy packages

apt-get install haproxy haproxyctl

Configuration

This setup uses host names instead of host addresses. Please note that this will require a restart of the HAProxy if the addresses change, as a lookup is only performed on startup.

/etc/haproxy.cfg

global
        log /dev/log    local0 info
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        maxconn 4000

defaults
        log     global
        option  tcplog
        option  dontlognull
        timeout server 5s
        timeout connect 5s
        timeout client 5s

frontend https_frontend
  bind *:443
  mode tcp

  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }

  # Different timeouts to account for the different use cases
  timeout server 5s
  timeout client 5s
  timeout client-fin 10s
  timeout connect 5s
  # this covers watches via kubectl etc.
  timeout tunnel 1h

  # This is a SNI lookup for 
  use_backend k8s_server if { req_ssl_sni -i <SOME_HOSTED_SSL_SERVICE } or { req_ssl_sni -i login.<YOUR_DOMAIN> }

  default_backend api_server


backend k8s_server
  mode tcp


  server mars1 YOUR_WORKER_NODE_1:443 check check-ssl verify none
  # repeat as needed for all nodes


backend api_server
  mode tcp
  balance source

  # Different timeouts to account for the different use cases
  timeout server 5s
  timeout client 5s
  timeout client-fin 10s
  timeout connect 5s
  # this covers watches via kubectl etc.
  timeout tunnel 1h

  server ctrl1 YOUR_API_SERVER_NODE_1:443 check check-ssl verify none
  server ctrl2 YOUR_API_SERVER_NODE_1:443 check check-ssl verify none
  server ctrl3 YOUR_API_SERVER_NODE_1:443 check check-ssl verify none