Skip to content

Latest commit

 

History

History
76 lines (50 loc) · 3.57 KB

README-deploy.md

File metadata and controls

76 lines (50 loc) · 3.57 KB
Raw

Building an environment

In order to test these features we need to deploy an infrastructure capable of replicating the asked features.

Setting up the network

For this environment we will be using a custom VNet with 2 subnets with non overlapping IP address spaces to emulate the connection. Since Express Route takes up to 40 minutes to deploy, we will not include how to set this up. But if guidance is needed on how to deploy connect the VNET gateways you can always go visit this repo or this repo for different clouds with OpenBSD.

Network topology

We will start this sample with a big address space provided by the On-prem network, we will reserve a segment explicit to the Azure services we will deploy, like AKS. Kubernetes will be configured with CNI plugin to take ownership of those IPs in the second address space. Lets do some calculations:

  1. On-Prem total adress space:

    • Addres Space: 10.0.0.0/8
    • HostMin: 10.0.0.1
    • HostMax: 10.255.255.254
    • Broadcast: 10.255.255.255
    • Hosts/Net: 16777214

  2. Azure services dedicated address space

    • Addres Space: 10.1.0.0/16
    • HostMin: 10.1.0.1
    • HostMax: 10.1.255.254
    • Broadcast: 10.1.255.255
    • Hosts/Net: 65534

Obviously this range is quite excessive since Azure CNI is limited to 16'000 configured IP Addesses. And advanced networking allows you to have 30 pods per node. So you can resize that if needed.

Let's also rememebet that the Kubernetes service IP address range:

  • Must not be within the VNet IP address range of your cluster
  • Must not overlap with any other VNets with which the cluster VNet peers
  • Must not overlap with any on-premises IPs
  • Kubernetes DNS service IP address: The IP address for the cluster's DNS service. This address must be within the Kubernetes service address range.
  • Docker Bridge address: The IP address and netmask to assign to the Docker bridge. This IP address must not be within the VNet IP address range of your cluster.

Comodo Certificates

In order to have a valid set of certificates for our registry, we will be using the free certificates from Comodo.

Deploying a daemonset to upload ca.crt to nodes

In order to trust the container registy we created, we need to allow tell docker to trust this new entity. Run the following command to do so"

cd daemonset-certs
kubectl apply -f upload-ca-cert-daemon.yml

Now, add the secret of the Container Registry to the kubernetes cluster:

kubectl create secret docker-registry myregistrykey --docker-server=https://<< DTR URL >>:<< DTR PORT >> --docker-username=<< DTR USERNAME >> --docker-password=<< DTR password >> --docker-email=<< A proper email value >>

Now, deploy a deployment and make sure that the secret is in place inside for the container reference:

imagePullSecrets:
- name: myregistrykey

On-prem registry

There are two registries we could use to emulate our on-prem registry scenario:

On-prem bamboo

Deploy AKS cluster