Skip to content

Commit

Permalink
Add example kubernetes deployment
Browse files Browse the repository at this point in the history
  • Loading branch information
nvtkaszpir committed Jul 11, 2024
1 parent 5a471cf commit 97d6972
Show file tree
Hide file tree
Showing 10 changed files with 287 additions and 0 deletions.
47 changes: 47 additions & 0 deletions docs/service.kubernetes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Running in kubernetes

Yes, because why not, especially if you run k3s :D

This is just an example but you should be able to adjust it to your needs.

## Overview

- each camera should be a separate kubernetes deployment, easier to manage.

- in `configs` there is a one file with env vars loaded per deployment,
those env vars **MUST BE** changed as in env vars
Also do not use double quotes the values.

- deployment possible using for example kustomize

## Examples

- [deployment-1.yaml](https://github.com/nvtkaszpir/prusa-connect-camera-script/k8s/deployment-1.yaml)
is an example to fetch image from a stream using ffmpeg and with custom
prusa-connect-camera.sh for easier development/iteration

- [deployment-2.yaml](https://github.com/nvtkaszpir/prusa-connect-camera-script/k8s/deployment-2.yaml)
is an example how to run it on Raspberry Pi with USB camera using default parameters.
You want to change `.spec.nodeName` and volumes to point to desired camera.

## More copies

If you want to add more cameras then you should:

- copy example config file from `config/` and adjust it
- copy desired deployment.yaml and adjust it
- update any `cam-x` references in the deployment.yaml
- update `.spec.nodeName` in the deployment.yaml
- if using direct camera update volumes - `dev-video` hostpath to point to the desired
camera device

- fine tune requests/limits to use less resources if needed

## Troubleshooting

- just look into the logs of the failing pods
- some hosts do not have predictable camera names, will have to think about udev
rules how to handle it...

- not tested with any device operators
- not tested with istio/traefik whatever.
4 changes: 4 additions & 0 deletions k8s/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# prusa-connect-camera-script in kubernetes

See [docs](https://nvtkaszpir.github.io/prusa-connect-camera-script/service.kubernetes/)
for more details.
7 changes: 7 additions & 0 deletions k8s/configs/cam-1.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
PRINTER_ADDRESS=
PRUSA_CONNECT_CAMERA_TOKEN=change-me
PRUSA_CONNECT_CAMERA_FINGERPRINT=change-me
CAMERA_SETUP_COMMAND=
CAMERA_DEVICE=/dev/null
CAMERA_COMMAND=ffmpeg
CAMERA_COMMAND_EXTRA_PARAMS=-y -i 'http://esp32-wrover-0461c8.intra.hlds.pl:8080/' -vframes 1 -q:v 1 -f image2 -update 1
5 changes: 5 additions & 0 deletions k8s/configs/cam-2.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
PRUSA_CONNECT_CAMERA_TOKEN=change-me
PRUSA_CONNECT_CAMERA_FINGERPRINT=change-me
CAMERA_DEVICE=/dev/video999
CAMERA_COMMAND=fswebcam
CAMERA_COMMAND_EXTRA_PARAMS=-S 10 --resolution 1280x720 --no-banner -s auto_exposure=1,brightness=128,contrast=5
96 changes: 96 additions & 0 deletions k8s/deployment-1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
apiVersion: apps/v1
kind: Deployment

metadata:
name: cam-1
labels:
app.kubernetes.io/name: cam-1
spec:
replicas: 1
revisionHistoryLimit: 3

strategy:
type: Recreate

selector:
matchLabels:
app.kubernetes.io/name: cam-1

template:
metadata:
labels:
app.kubernetes.io/name: cam-1

spec:
automountServiceAccountToken: false
enableServiceLinks: false
containers:
- name: script
tty: true # get that logs flowing to stdout/stderr
command:
- /script/prusa-connect-camera.sh
envFrom:
- secretRef:
name: cam-1
image: quay.io/kaszpir/prusa-connect-script:latest
imagePullPolicy: IfNotPresent
securityContext:
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
# add:
# - NET_RAW
ports:
- name: status
containerPort: 8080
protocol: TCP

resources:
requests:
cpu: "0.1"
memory: "32Mi"
limits:
cpu: "1.0"
memory: "128Mi"

lifecycle:
preStop:
exec:
command: ["/bin/sh","-c","echo graceful-shutdown-start;sleep 2;echo graceful-shutdown-end"] # wait for inflight requests to finish

livenessProbe:
exec:
command:
- bash
- -c
- test $(find /dev/shm/ -mmin -1 | wc -l) -gt "0"
initialDelaySeconds: 5
periodSeconds: 15

# readinessProbe:
# httpGet:
# path: /health
# port: status

volumeMounts:
- mountPath: /script/
name: script
- mountPath: /dev/shm
name: dev-shm

terminationGracePeriodSeconds: 10
volumes:

- name: dev-shm
emptyDir:
medium: Memory

- name: script
configMap:
name: script
defaultMode: 0755
90 changes: 90 additions & 0 deletions k8s/deployment-2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
---
apiVersion: apps/v1
kind: Deployment

metadata:
name: cam-2
labels:
app.kubernetes.io/name: cam-2
spec:
replicas: 1
revisionHistoryLimit: 3

strategy:
type: Recreate

selector:
matchLabels:
app.kubernetes.io/name: cam-2

template:
metadata:
labels:
app.kubernetes.io/name: cam-2

spec:
nodeName: hormex # change this to the desired host with given hardware
automountServiceAccountToken: false
enableServiceLinks: false
containers:
- name: script
tty: true # get that logs flowing to stdout/stderr
envFrom:
- secretRef:
name: cam-2
image: quay.io/kaszpir/prusa-connect-script:latest
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command: ["/bin/sh","-c","echo graceful-shutdown-start;sleep 2;echo graceful-shutdown-end"] # wait for inflight requests to finish
livenessProbe:
exec:
command:
- bash
- -c
- test $(find /dev/shm/ -mmin -1 | wc -l) -gt "0"
initialDelaySeconds: 5
periodSeconds: 15

securityContext:
# allowPrivilegeEscalation: false
privileged: true
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
capabilities:
drop:
- ALL
# add:
# - NET_RAW # needed for ping if not using

resources:
requests:
cpu: "0.1"
memory: "32Mi"
limits:
cpu: "1.0"
memory: "128Mi"

volumeMounts:
- mountPath: /dev/shm
name: dev-shm
- mountPath: /dev/video999 # must be the same as in `volumes` section and in cam.env
name: dev-video
# securityContext:
# runAsUser: 1000
# # runAsGroup: video
# # fsGroup: 2000
# tolerations:
# - key: "rpi"
# operator: "Exists"
# effect: "NoSchedule"
terminationGracePeriodSeconds: 10 # let it finish the sending the picture
volumes:
- name: dev-shm
emptyDir:
medium: Memory
- name: dev-video
hostPath:
path: /dev/video1 # change this to the device on the host
29 changes: 29 additions & 0 deletions k8s/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prusa-connect-camera

resources:
- namespace.yaml

- deployment-1.yaml
# - deployment-2.yaml

secretGenerator:
- name: cam-1
envs:
- configs/cam-1.env
- name: cam-2
envs:
- configs/cam-2.env


configMapGenerator:
- name: script
files:
- prusa-connect-camera.sh

images:
- name: quay.io/kaszpir/prusa-connect-script
newName: bagno.hlds.pl:16000/quay.io/kaszpir/prusa-connect-script
newTag: "5a471cf"
7 changes: 7 additions & 0 deletions k8s/namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: prusa-connect-camera
name: prusa-connect-camera
1 change: 1 addition & 0 deletions k8s/prusa-connect-camera.sh
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ nav:
- Overview: service.md
- Systemd: service.systemd.md
- Docker: service.docker.md
- Kubernetes: service.kubernetes.md

- Configuration Tuning: configuration.tuning.md
- Configuration all env vars: configuration.env.full.md
Expand Down

0 comments on commit 97d6972

Please sign in to comment.