Add Elixir WebRTC - Nexus app demo to docs

l7mp · Sep 25, 2024 · 1140389 · 1140389
1 parent c9c2c2e
commit 1140389
Show file tree

Hide file tree

Showing 6 changed files with 370 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -576,6 +576,12 @@ applications into Kubernetes.
   behind STUNner. The docs also show how to obtain a valid TLS certificate to secure your signaling
   connections, using [cert-manager](https://cert-manager.io), [nip.io](https://nip.io) and [Let's
   Encrypt](https://letsencrypt.org).
+* [Video-conferencing with Elixir WebRTC](/docs/examples/elixir-webrtc/README.md): This tutorial helps 
+  you deploy a fully fledged [Elixir WebRTC](https://elixir-webrtc.org/) video-conferencing room called 
+  [Nexus](https://github.com/elixir-webrtc/apps/tree/master/nexus) into Kubernetes
+  behind STUNner. The docs also show how to obtain a valid TLS certificate to secure your signaling
+  connections, using [cert-manager](https://cert-manager.io), [nip.io](https://nip.io) and [Let's
+  Encrypt](https://letsencrypt.org).
 * [Video-conferencing with Jitsi](/docs/examples/jitsi/README.md): This tutorial helps you deploy a
   fully fledged [Jitsi](https://jitsi.org) video-conferencing service into Kubernetes behind
   STUNner. The docs also show how to obtain a valid TLS certificate to secure your signaling

diff --git a/docs/README.md b/docs/README.md
@@ -41,6 +41,7 @@
 * [Video-conferencing with LiveKit](examples/livekit/README.md)
 * [Video-conferencing with mediasoup](examples/mediasoup/README.md)
 * [Video-conferencing with Janus](examples/janus/README.md)
+* [Video-conferencing with Elixir WebRTC](examples/elixir-webrtc/README.md)
 * [Video-conferencing with Jitsi](examples/jitsi/README.md)
 * [Cloud-gaming with CloudRetro](examples/cloudretro/README.md)
 * [Remote desktop access with Neko](examples/neko/README.md)

diff --git a/docs/examples/elixir-webrtc/README.md b/docs/examples/elixir-webrtc/README.md
@@ -0,0 +1,211 @@
+# STUNner demo: Video-conferencing with Nexus
+
+This document guides you through the installation of [Elixir WebRTC](https://elixir-webrtc.org/), called [Nexus](https://github.com/elixir-webrtc/apps/tree/master/nexus) into Kubernetes, when it is used together with the STUNner WebRTC media gateway.
+
+In this demo you will learn to:
+
+- integrate a typical WebRTC application with STUNner,
+- obtain a valid TLS certificate to secure the signaling plane,
+- deploy the Nexus app server into Kubernetes, and
+- configure STUNner to expose Nexus to clients.
+
+## Prerequisites
+
+The below installation instructions require an operational cluster running a supported version of Kubernetes (>1.22). Most hosted or private Kubernetes cluster services will work, but make sure that the cluster comes with a functional load-balancer integration (all major hosted Kubernetes services should support this). Otherwise, STUNner will not be able to allocate a public IP address for clients to reach your WebRTC infra. As a regrettable exception, Minikube is unfortunately not supported for this demo. The reason is that [Let's Encrypt certificate issuance is not available with nip.io](https://medium.com/@EmiiKhaos/there-is-no-possibility-that-you-can-get-lets-encrypt-certificate-with-nip-io-7483663e0c1b); later on you will learn more about why this is crucial above.
+
+## Installation
+
+> [!NOTE]
+>
+> Let's start with a disclaimer. The Nexus client-side application must work over a secure HTTPS connection, because [getUserMedia](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia#browser_compatibility) is available only in secure contexts. This implies that the client-server signaling connection must be secure too. In this demo, we will aim to obtain a proper CA-signed certificate (self-signed certificates haven't been tested). Obtaining a valid TLS certificate is a challenge. Thus, the majority of the below installation guide will be about securing client connections to Nexus over TLS; as it turns out, once HTTPS is correctly working integrating Nexus with STUNner is very simple.
+
+In the below example, STUNner will be installed into the identically named namespace (`stunner`), while Nexus and the Ingress gateway will live in the `default` namespace.
+
+### TLS certificates
+
+As mentioned above, the Nexus WebRTC server will need a valid TLS cert, which means it must run behind an existing DNS domain name backed by a CA signed TLS certificate. This is simple if you have your own domain, but if you don't then [nip.io](https://nip.io) provides a dead simple wildcard DNS for any IP address. We will use this to "own a domain" and obtain a CA signed certificate for Nexus. This will allow us to point the domain name `client-<ingress-IP>.nip.io` to an ingress HTTP gateway in our Kubernetes cluster, which will then use some automation (namely, cert-manager) to obtain a valid CA signed cert.
+
+Note that public wildcard DNS domains might run into [rate limiting](https://letsencrypt.org/docs/rate-limits/) issues. If this occurs you can try [alternative services](https://moss.sh/free-wildcard-dns-services/) instead of `nip.io`.
+
+### Ingress
+
+The first step of obtaining a valid cert is to install a Kubernetes Ingress: this will be used during the validation of our certificates and to terminate client TLS encrypted contexts.
+
+Install an ingress controller into your cluster. We used the official [nginx ingress](https://github.com/kubernetes/ingress-nginx), but this is not required.
+
+```console
+helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
+helm repo update
+helm install ingress-nginx ingress-nginx/ingress-nginx
+```
+
+Wait until Kubernetes assigns an external IP to the Ingress.
+
+```console
+until [ -n "$(kubectl get service ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" ]; do sleep 1; done
+```
+
+Store the Ingress IP address Kubernetes assigned to our Ingress; this will be needed later when we configure the validation pipeline for our TLS certs.
+
+```console
+kubectl get service ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
+export INGRESSIP=$(kubectl get service ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
+export INGRESSIP=$(echo $INGRESSIP | sed 's/\./-/g')
+```
+
+### Cert manager
+
+We use the official [cert-manager](https://cert-manager.io) to automate TLS certificate management.
+
+Add the Helm repository, which contains the cert-manager Helm chart, and install the charts:
+
+```console
+helm repo add cert-manager https://charts.jetstack.io
+helm repo update
+helm install cert-manager jetstack/cert-manager --namespace cert-manager \
+    --create-namespace --set global.leaderElection.namespace=cert-manager \
+    --set crds.enabled=true --timeout 600s
+```
+
+At this point we have all the necessary boilerplate set up to automate TLS issuance for Nexus.
+
+### STUNner
+
+Now comes the fun part. The simplest way to run this demo is to clone the [STUNner git repository](https://github.com/l7mp/stunner) and deploy (after some minor modifications) the [manifest](nexus-server.yaml) packaged with STUNner.
+
+Install the STUNner gateway operator and STUNner via [Helm](https://github.com/l7mp/stunner-helm):
+
+```console
+helm repo add stunner https://l7mp.io/stunner
+helm repo update
+helm install stunner-gateway-operator stunner/stunner-gateway-operator --create-namespace --namespace=stunner
+```
+
+Configure STUNner to act as a STUN/TURN server to clients, and route all received media to the Nexus pods.
+
+```console
+git clone https://github.com/l7mp/stunner
+cd stunner
+kubectl apply -f docs/examples/elixir-webrtc/nexus-call-stunner.yaml
+```
+
+The relevant parts here are the STUNner [Gateway definition](../../GATEWAY.md#gateway), which exposes the STUNner STUN/TURN server over UDP:3478 to the Internet, and the [UDPRoute definition](../../GATEWAY.md#udproute), which takes care of routing media to the pods running the Nexus Gateway service.
+
+```yaml
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: udp-gateway
+  namespace: stunner
+spec:
+  gatewayClassName: stunner-gatewayclass
+  listeners:
+    - name: udp-listener
+      port: 3478
+      protocol: UDP
+---
+apiVersion: stunner.l7mp.io/v1
+kind: UDPRoute
+metadata:
+  name: nexus
+  namespace: stunner
+spec:
+  parentRefs:
+    - name: udp-gateway
+  rules:
+    - backendRefs:
+        - kind: Service
+          name: nexus
+          namespace: default
+```
+
+Once the Gateway resource is installed into Kubernetes, STUNner will create a Kubernetes LoadBalancer for the Gateway to expose the TURN server on UDP:3478 to clients. It can take up to a minute for Kubernetes to allocate a public external IP for the service.
+
+Wait until Kubernetes assigns an external IP and store the external IP assigned by Kubernetes to
+STUNner in an environment variable for later use.
+
+```console
+until [ -n "$(kubectl get svc udp-gateway -n stunner -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" ]; do sleep 1; done
+export STUNNERIP=$(kubectl get service udp-gateway -n stunner -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
+```
+### Nexus ocker images
+
+The crucial step of integrating *any* WebRTC media server with STUNner is to ensure that the server instructs the clients to use STUNner as the STUN/TURN server. 
+Unfortunately, currently the [official Nexus Docker image](ghcr.io/elixir-webrtc/apps/nexus) does not support this configuration in runtime (by default Google's STUN server is hardcoded into it).
+Therefore, we have to modify this setting to STUNner's IP and build a new Docker image.
+
+In order to achieve this, first clone the Elixir WebRTC sample app repository:
+
+```console
+git clone https://github.com/elixir-webrtc/apps/
+cd apps/nexus
+```
+
+You have to modify the ICE config to use STUNner with the given credentials in two files:
+
+`assets/js/home.js`:
+```
+...
+# const pcConfig = { iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] };
+
+# change to:
+
+  const pcConfig = { iceServers: [{ urls: 'turn:<STUNNERIP>:3478?transport=udp', username: 'user-1', credential: 'pass-1',iceTransportPolicy: 'relay'}] };
+```
+
+`lib/nexus/peer.ex`:
+```
+...
+# ice_servers: [%{urls: "stun:stun.l.google.com:19302"}],
+
+# change to:
+
+  ice_servers: [%{urls: "turn:<STUNNERIP>:3478?transport=udp", "username": "user-1", "credential": "pass-1", "iceTransportPolicy": "relay"}],
+```
+
+Now rebuild the Docker image, and push it into your image repository:
+```
+export MYREPO=myrepo # use you'r own Docker repository name!
+sudo docker build -t $MYREPO/nexus .
+sudo docker push $MYREPO/nexus
+```
+
+After uploading the image, you also have to modify the Nexus image repo location in the Kubernetes deployment file.
+
+```console
+sed -i "s/l7mp/$MYREPO/g" docs/examples/elixir-webrtc/nexus-server.yaml
+```
+
+We also need the Ingress external IP address we have stored previously: this will make sure that the TLS certificate created by cert-manager will be bound to the proper `nip.io` domain and IP address.
+
+```console
+sed -i "s/ingressserviceip/$INGRESSIP/g" docs/examples/elixir-webrtc/nexus-server.yaml
+```
+
+Finally, fire up Nexus.
+
+```console
+kubectl apply -f docs/examples/elixir-webrtc/nexus-server.yaml
+```
+
+The demo installation bundle includes a few resources to deploy Nexus:
+
+- Nexus deployment and service,
+- a cluster issuer for the TLS certificates,
+- an Ingress resource to terminate the secure connections between your browser and the Kubernetes cluster.
+
+Wait until all pods become operational and jump right into testing!
+
+## Test
+
+After installing everything, execute the following command to retrieve the URL of your freshly deployed Nexus demo app:
+
+```console
+echo INGRESSIP.nip.io
+```
+
+Copy the URL into your browser, and if everything is set up correctly, you should be able to connect to a video room. If you repeat the procedure in a separate browser tab you can enjoy a nice video-conferencing session with yourself, with the twist that all media between the browser tabs is flowing through STUNner and the Nexus server deployed in you Kubernetes cluster.
+
+# Help
+
+STUNner development is coordinated in Discord, feel free to [join](https://discord.gg/DyPgEsbwzc).
diff --git a/docs/examples/elixir-webrtc/nexus-call-stunner.yaml b/docs/examples/elixir-webrtc/nexus-call-stunner.yaml
@@ -0,0 +1,51 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: GatewayClass
+metadata:
+  name: stunner-gatewayclass
+spec:
+  controllerName: "stunner.l7mp.io/gateway-operator"
+  parametersRef:
+    group: "stunner.l7mp.io"
+    kind: GatewayConfig
+    name: stunner-gatewayconfig
+    namespace: stunner
+  description: "STUNner is a WebRTC ingress gateway for Kubernetes"
+
+---
+apiVersion: stunner.l7mp.io/v1
+kind: GatewayConfig
+metadata:
+  name: stunner-gatewayconfig
+  namespace: stunner
+spec:
+  realm: stunner.l7mp.io
+  authType: static
+  userName: "user-1"
+  password: "pass-1"
+
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: udp-gateway
+  namespace: stunner
+spec:
+  gatewayClassName: stunner-gatewayclass
+  listeners:
+    - name: udp-listener
+      port: 3478
+      protocol: TURN-UDP
+---
+apiVersion: stunner.l7mp.io/v1
+kind: UDPRoute
+metadata:
+  name: nexus
+  namespace: stunner
+spec:
+  parentRefs:
+    - name: udp-gateway
+  rules:
+    - backendRefs:
+        - kind: Service
+          name: nexus
+          namespace: default
diff --git a/docs/examples/elixir-webrtc/nexus-server.yaml b/docs/examples/elixir-webrtc/nexus-server.yaml
@@ -0,0 +1,100 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: nexus
+  labels:
+    app.kubernetes.io/name: nexus
+spec:
+  type: ClusterIP
+  ports:
+    - port: 4000
+      targetPort: 4000
+      protocol: TCP
+      name: http
+  selector:
+    app.kubernetes.io/name: nexus
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nexus
+  labels:
+    app.kubernetes.io/name: nexus
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: nexus
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: nexus
+    spec:
+      terminationGracePeriodSeconds: 18000 # 5 hours
+      containers:
+        - name: nexus
+          image: "l7mp/nexus"
+          imagePullPolicy: IfNotPresent
+          env:
+          - name: SECRET_KEY_BASE
+            value: f0e01531666a6ccd5b15879a3c00a15f8feaaea86230cf35d96fe0b8e468bc553549e47edb737a4d1530d1c56a3a7a3827443e1bd00aa46adb44dc6aa087ff9f
+          - name: PHX_SERVER
+            value: "1"
+          - name: PHX_HOST
+            value: ingressserviceip.nip.io
+          - name: ADMIN_USERNAME
+            value: admin
+          - name: ADMIN_PASSWORD
+            value: admin
+          - name: ICE_PORT_RANGE
+            value: 62000-63000
+          - name: NEXUS_POD_IP
+            valueFrom:
+              fieldRef:
+                fieldPath: status.podIP
+          ports:
+            - name: http
+              containerPort: 4000
+              protocol: TCP
+---
+# Ingress for both Nexus
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: nexus
+  annotations:
+    cert-manager.io/cluster-issuer: "letsencrypt-prod"
+    nginx.ingress.kubernetes.io/ssl-redirect: "true"
+spec:
+  ingressClassName: nginx
+  tls:
+  - hosts:
+      - ingressserviceip.nip.io
+    secretName: nexus-secret-tls
+  rules:
+  - host: ingressserviceip.nip.io
+    http:
+      paths:
+      - path: /
+        pathType: Prefix
+        backend:
+          service:
+            name: nexus
+            port:
+              number: 4000
+---
+apiVersion: cert-manager.io/v1
+kind: ClusterIssuer
+metadata:
+  name: letsencrypt-prod
+spec:
+  acme:
+    email: info@l7mp.io
+    server: https://acme-v02.api.letsencrypt.org/directory
+    privateKeySecretRef:
+      name: letsencrypt-secret-prod
+    solvers:
+    - http01:
+        ingress:
+          class: nginx
+---
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -25,6 +25,7 @@ nav:
   - 'Kurento Magic Mirror': 'examples/kurento-magic-mirror/README.md'
   - 'LiveKit': 'examples/livekit/README.md'
   - 'Janus': 'examples/janus/README.md'
+  - 'Elixir WebRTC': 'examples/elixir-webrtc/README.md'
   - 'Jitsi': 'examples/jitsi/README.md'
   - 'mediasoup': 'examples/mediasoup/README.md'
   - 'CloudRetro': 'examples/cloudretro/README.md'