README.md|example: improve docs and examples

This commit:

* Moves metallb.conf to test/cni
* Refactors the example manifests. Adds a simple client to showcase
  affinity
* Adds an example/k8s/README.md
* Cleans up README.md (more cleanings to come).

README.md

@@ -7,7 +7,7 @@ affinity across all _ports_ within the service.
 
 See the original use-case [here](docs/use-cases/network-telemetry-nfacctd.md).
 
-:warning: While `sfunnel` should be fully functional, and has been [validated](#tested-environments),
+:warning: While `sfunnel` should be fully functional, and has been [validated](#support),
 it is still in an early development stage.
 
 ## At a glance
@@ -69,6 +69,20 @@ container along with the [rules](docs/rules.md) in `SFUNNEL_RULESET`:
 +              add: [BPF, NET_ADMIN, SYS_ADMIN]
 +          volumeMounts:
 +            - name: bpffs
++              mountPath: /sys/fs/bpf
++        - name: sfunnel-init-egress
++          env:
++            - name: SFUNNEL_RULESET
++              value: ip tcp sport 8080 actions funnel tcp dport 540 sport 80
++            - name: DIRECTION
++              value: egress
++          image: ghcr.io/datahangar/sfunnel:0.0.11@sha256:5f130c2bfc95fb0d264ad54c52b1fef26c58e5635f11b8b862efe611b98b1f9a
++          securityContext:
++            privileged: false #Set to true for some public clouds (e.g. GKE standard)
++            capabilities:
++              add: [BPF, NET_ADMIN, SYS_ADMIN]
++          volumeMounts:
++            - name: bpffs
 +              mountPath: /sys/fs/bpf
          - name: nginx
            image: nginx:latest
@@ -80,21 +94,40 @@ container along with the [rules](docs/rules.md) in `SFUNNEL_RULESET`:
 +         hostPath:
 +           path: /sys/fs/bpf
 ```
-(_Note: funneling HTTPs `TCP/443` through `TCP/80` would work the same way. Manifest
+(_funneling HTTPs `TCP/443` through `TCP/80` would work the same way. Manifest
 is just too long for this example_)
 
 On the other end (e.g. a Linux host, server etc..), deploy it with the
 matching [rules](docs/rules.md):
 
 ```shell
-SFUNNEL_RULESET="ip daddr <your LB IP1> tcp port 443 actions funnel tcp dport 80 sport 540;\
-  ip daddr <your LB IP1> tcp port 8080 actions funnel tcp dport 80 sport 540"
-docker run --network="host" --privileged -e SFUNNEL_RULESET="$SFUNNEL_RULESET" sfunnel
+IFACES=eth0 LB_IP=1.1.1.1 \
+SFUNNEL_RULESET="ip daddr ${LB_IP} tcp dport 8080 actions funnel tcp dport 80 sport 540" \
+docker run --privileged --network=host -it -e IFACES -e DIRECTION="egress" -e SFUNNEL_RULESET ghcr.io/datahangar/sfunnel:0.0.11
+```
+
+```shell
+IFACES=eth0 LB_IP=1.1.1.1 \
+SFUNNEL_RULESET="ip saddr ${LB_IP} tcp sport 80 dport 540 actions unfunnel tcp" \
+docker run --privileged --network=host -it -e IFACES -e DIRECTION="ingress" -e SFUNNEL_RULESET ghcr.io/datahangar/sfunnel:0.0.11
 ```
 
 The `sfunnel` container will run, load the eBPF code and finish its execution.
 
-## Tested environments
+## Support
+
+### Service types
+
+* `ClusterIP`: supported
+* `LoadBalancer`: supported
+* `NodePort`: _untested, but should work_
+
+> :pencil: **Note**
+>
+> Currently `internalTrafficPolicy: Local` for `ClusterIP` and
+> `externalTrafficPolicy: Local` for `NodePort` and `LoadBalancer` services are required.
+
+### Environments
 
 * **Google Kubernetes Engine(GKE)**: Standard cluster.
    - Autopilot clusters are _not supported_ due to lack of eBPF support.

example/k8s/README.md

@@ -0,0 +1,56 @@
+# K8s example
+
+This example showcases how to provide multi-port/flow affinity between
+two HTTP services, one in HTTP port 80 and another in 8080.
+
+Apply the web server and service manifests:
+
+```
+kubectl apply -k .
+```
+The web server returns its hostname (pod name) as part of the
+HTTP GET response.
+
+## Testing the LoadBalancer service
+
+For affinity to work, traffic needs to be intercepted before it hits the LoadBalancer.
+
+Load the following sfunnel programs to your egress interface (e.g. `eth0`)
+
+##### Ingress:
+
+```
+IFACES=eth0 LB_IP=1.1.1.1 \
+SFUNNEL_RULESET="ip daddr ${LB_IP} tcp dport 8080 actions funnel tcp dport 80 sport 540" \
+docker run --privileged --network=host -it -e IFACES -e DIRECTION="egress" -e SFUNNEL_RULESET ghcr.io/datahangar/sfunnel:0.0.11
+```
+##### Egress
+
+
+```
+IFACES=eth0 LB_IP=1.1.1.1 \
+SFUNNEL_RULESET="ip saddr ${LB_IP} tcp sport 80 dport 540 actions unfunnel tcp" \
+docker run --privileged --network=host -it -e IFACES -e DIRECTION="ingress" -e SFUNNEL_RULESET ghcr.io/datahangar/sfunnel:0.0.11
+```
+
+##### Cleanup
+
+You detach both programs by running:
+
+```
+IFACES=eth0 \
+docker run --privileged --network=host -it -e IFACES -e DIRECTION="both" -e CLEAN=1 ghcr.io/datahangar/sfunnel:0.0.11
+```
+
+## Testing the ClusterIP service
+
+Apply the `client.yaml` (not part of kustomize):
+
+```
+kubectl apply -f client.yaml
+```
+
+`client.yaml` contains a simple script that `curl`s both HTTP/80 and
+HTTP/8080 and verifies that both hostnames match.
+
+:warning: Please note the [known issues](#) with Cilium in non-KPR mode

example/k8s/client.yaml

@@ -73,8 +73,6 @@ spec:
           image: ghcr.io/datahangar/sfunnel:0.0.11@sha256:5f130c2bfc95fb0d264ad54c52b1fef26c58e5635f11b8b862efe611b98b1f9a
           imagePullPolicy: IfNotPresent
           env:
-            - name: DEBUG
-              value: "1"
             - name: SFUNNEL_RULESET
               value: ip tcp sport 80 dport 540 actions unfunnel tcp
           securityContext:
@@ -88,8 +86,6 @@ spec:
           image: ghcr.io/datahangar/sfunnel:0.0.11@sha256:5f130c2bfc95fb0d264ad54c52b1fef26c58e5635f11b8b862efe611b98b1f9a
           imagePullPolicy: IfNotPresent
           env:
-            - name: DEBUG
-              value: "1"
             - name: DIRECTION
               value: egress
             - name: SFUNNEL_RULESET

example/k8s/kustomization.yaml

@@ -2,6 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 
 resources:
-  - metallb-conf.yaml
   - nginx.yaml
   - service.yaml

example/k8s/nginx.yaml

@@ -41,8 +41,6 @@ spec:
           image: ghcr.io/datahangar/sfunnel:0.0.11@sha256:5f130c2bfc95fb0d264ad54c52b1fef26c58e5635f11b8b862efe611b98b1f9a
           imagePullPolicy: IfNotPresent
           env:
-            - name: DEBUG
-              value: "1"
             - name: SFUNNEL_RULESET
               value: ip tcp dport 80 sport 540 actions unfunnel tcp
           securityContext:
@@ -56,8 +54,6 @@ spec:
           image: ghcr.io/datahangar/sfunnel:0.0.11@sha256:5f130c2bfc95fb0d264ad54c52b1fef26c58e5635f11b8b862efe611b98b1f9a
           imagePullPolicy: IfNotPresent
           env:
-            - name: DEBUG
-              value: "1"
             - name: SFUNNEL_RULESET
               value: ip tcp sport 8080 actions funnel tcp dport 540 sport 80
             - name: DIRECTION

test/cni/Makefile

@@ -36,6 +36,8 @@ _start_minikube:
 			while [[ "$$(minikube kubectl -- -n kube-system get pods | grep -v NAME | grep -v Running)" != "" ]]; do sleep 1; done; \
 			echo "All running!"; \
 		 fi
+	$(QUIET) minikube kubectl -- apply -f metallb-conf.yaml
+
 _stop_minikube:
 	$(QUIET) minikube delete || true
 

test/cni/example-patch.yaml

@@ -4,3 +4,14 @@ metadata:
   name: my-nginx-deployment
 spec:
   replicas: 12
+  template:
+    spec:
+      initContainers:
+        - name: sfunnel-init
+          env:
+            - name: DEBUG
+              value: "1"
+        - name: sfunnel-init-egress
+          env:
+            - name: DEBUG
+              value: "1"