-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v0.3.0 Test Matrix #43
Comments
KindRecording: https://asciinema.org/a/SvNWIgRo0xdKQEkCj0cyQ293b Prerequisites
Setup cluster# Crate cluster
kind create cluster --image kindest/node:v1.28.0
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached.. ❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
lunatic-demo-6475554875-zbrwh 2/2 Running 0 73s
wasm-slight-6467bcc5bc-gx29z 2/2 Running 0 73s
wasm-spin-74c4cf5c77-4slq4 2/2 Running 0 73s
wasm-wws-888f6bc4b-qj6rx 2/2 Running 0 73s
wasmedge-demo-5ff758d79-b5v6j 2/2 Running 0 73s
wasmer-demo-857f947cb7-8qx5g 1/2 Terminating 2 73s
wasmer-demo-857f947cb7-xqw4s 2/2 Running 0 12s
wasmtime-demo-56c78ddd95-x6wmc 2/2 Running 0 73s |
ColimaRecording: https://asciinema.org/a/nF5m7fHgdr3Bq8Qdj1scy66Jq Prerequisites
Setup cluster# Crate cluster
colima start -k -r containerd --kubernetes-version v1.28.1+k3s1
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached.
Only runwasi shims become ready: ❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
wasmer-demo-857f947cb7-lr7rk 2/2 Running 0 2m39s
wasmtime-demo-56c78ddd95-8jsbz 2/2 Running 0 2m39s
wasmedge-demo-5ff758d79-7v4rx 2/2 Running 0 2m39s
wasm-slight-6467bcc5bc-pqt2x 0/2 CrashLoopBackOff 8 (47s ago) 2m39s
wasm-wws-888f6bc4b-gnzz4 0/2 CrashLoopBackOff 8 (43s ago) 2m39s
lunatic-demo-6475554875-wqphh 0/2 CrashLoopBackOff 8 (30s ago) 2m39s
wasm-spin-74c4cf5c77-m5z5x 0/2 CrashLoopBackOff 3 (11s ago) 2m39s |
Namespace.soRecording: https://asciinema.org/a/QnreeaoqsldJWUi6ULsvpykT0 Prerequisites
Setup clustercat > kwasm.json << EOF
{"disks":[{"name":"kwasm","image":"ghcr.io/kwasm/kwasm-node-installer:pr-42"}],"containerd_shims":[{"name":"lunatic","runtime_type":"/disk/kwasm/assets/containerd-shim-lunatic-v1"},{"name":"slight","runtime_type":"/disk/kwasm/assets/containerd-shim-slight-v1"},{"name":"spin","runtime_type":"/disk/kwasm/assets/containerd-shim-spin-v1"},{"name":"wws","runtime_type":"/disk/kwasm/assets/containerd-shim-wws-v1"},{"name":"wasmedge","runtime_type":"/disk/kwasm/assets/containerd-shim-wasmedge-v1"},{"name":"wasmer","runtime_type":"/disk/kwasm/assets/containerd-shim-wasmer-v1"},{"name":"wasmtime","runtime_type":"/disk/kwasm/assets/containerd-shim-wasmtime-v1"}]}
EOF
nsc create --experimental_from kwasm.json
nsc kubeconfig write 1rq3tlt0mfpu6
export KUBECONFIG=~/Library/Caches/ns/tmp/kubeconfig/973245096.yaml Test executionNamespace.so clusters can't restart containerd. When containerd is terminated the whole cluster is terminated. But they have a parameter to customize containerd config before startup. kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached. ❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
wasmer-demo-558c75c49f-p7sf6 2/2 Running 0 30s
wasmtime-demo-6979d575-48nq4 2/2 Running 0 30s
lunatic-demo-767757cf9-2x55j 2/2 Running 0 30s
wasmedge-demo-7fb59fffdc-46k4l 2/2 Running 0 30s
wasm-spin-85bfb69795-8q877 2/2 Running 0 30s
wasm-slight-5646fd7cd5-swtft 2/2 Running 1 (26s ago) 30s
wasm-wws-c875b9664-wf84x 0/2 Terminating 0 29s
wasm-wws-c875b9664-8btjt 2/2 Running 0 5s |
Would you please link the issue (if the issue was not created, raise an issue to upstream) here for reference? |
Oh no, the symbol stripping work @jprendes did might help to mitigate the disk pressure issue! https://cloud-native.slack.com/archives/C04LTPB6Z0V/p1695201699156729. |
Thank you for conducting tests on the shims on multiple k8s distributions; your findings are invaluable for us maintainers to understand our operational landscape and potential challenges with each release. To automate this process, I suggest implementing integration tests via GitHub Actions within the upstream repositories, runwasi and containerd-wasm-shims. This would serve as a robust safeguard for future releases. The challenge lies in, for one, writing tests for two repositories and, for two, accommodating the numerous Kubernetes distributions, as you listed a table. Despite these complexities, I'm willing to dedicate some time to the automation effort for both the runwasi and containerd-wasm-shims repositories. Please note that I won't have much capacity until I come back from my vacation 🏝️ |
@Mossaka I don't expect you to do anything while you're on vacation, not even read github issues 😉. |
AWS EKSPrerequisites
Setup cluster# Crate cluster
eksctl create cluster --version 1.27
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached.. ❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
lunatic-demo-785446bf5-sfhmw 2/2 Running 0 58s
wasm-slight-9566467b7-ntncd 2/2 Running 0 58s
wasm-spin-8cc64fd85-vvr97 2/2 Running 0 58s
wasm-wws-ff448f95d-gz7qk 2/2 Running 0 58s
wasmedge-demo-774b4489c5-z7blc 2/2 Running 0 59s
wasmer-demo-5b7c886887-pt7ds 2/2 Running 0 59s
wasmtime-demo-6878c5447-r46d8 2/2 Running 0 58s |
CIVO K3sPrerequisites
Setup cluster# Crate cluster
civo kubernetes create kwasm
civo kubernetes config kwasm --save
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached.. ❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
wasm-slight-5646fd7cd5-q7mgl 2/2 Running 0 3m7s
wasmer-demo-558c75c49f-p9sq7 2/2 Running 0 3m7s
wasm-wws-c875b9664-s9b57 2/2 Running 0 3m7s
wasmedge-demo-7fb59fffdc-jqm2s 2/2 Running 0 3m7s
wasmtime-demo-6979d575-zs2hl 2/2 Running 0 3m7s
lunatic-demo-767757cf9-8r88m 2/2 Running 0 3m7s
wasm-spin-85bfb69795-cr6nv 2/2 Running 0 3m7s |
ColimaPrerequisites
Setup cluster# Crate cluster
colima start -k -r containerd --kubernetes-version v1.28.1+k3s1 --network-address
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached.
❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
wasm-slight-6467bcc5bc-dshjf 2/2 Running 0 2m3s
wasm-spin-74c4cf5c77-zc2pv 2/2 Running 0 2m3s
wasm-wws-888f6bc4b-lj8nh 2/2 Running 0 2m3s
lunatic-demo-6475554875-kv497 2/2 Running 0 2m3s
wasmedge-demo-5ff758d79-b92kt 2/2 Running 0 2m3s
wasmer-demo-857f947cb7-82kpn 2/2 Running 0 2m3s
wasmtime-demo-56c78ddd95-zmqhp 2/2 Running 0 2m3s |
DigitalOcean KubernetesPrerequisites
Setup cluster# Crate cluster
doctl kubernetes cluster create kwasm
doctl kubernetes cluster kubeconfig save kwasm
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached. ❯ kubectl get pod
NAME READY STATUS RESTARTS AGE
lunatic-demo-6475554875-fxrr4 2/2 Running 0 49s
wasm-slight-6467bcc5bc-jbkkg 2/2 Running 0 49s
wasm-spin-74c4cf5c77-22zkt 2/2 Running 0 49s
wasm-wws-888f6bc4b-xd8k5 2/2 Running 0 48s
wasmedge-demo-5ff758d79-r6h9m 2/2 Running 0 49s
wasmer-demo-857f947cb7-rsrzz 2/2 Running 0 49s
wasmtime-demo-56c78ddd95-j4cwb 2/2 Running 0 49s |
Killercoda.comPrerequisites
Setup clustervisit: https://killercoda.com/playgrounds/scenario/kubernetes # Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached. NAME READY STATUS RESTARTS AGE
lunatic-demo-785446bf5-zjg2w 2/2 Running 0 2m2s
wasm-slight-9566467b7-jcgjd 2/2 Running 0 2m2s
wasm-spin-8cc64fd85-zc5d2 2/2 Running 0 2m2s
wasm-wws-ff448f95d-lpx4n 2/2 Running 0 2m2s
wasmedge-demo-774b4489c5-bgdwf 2/2 Running 0 2m2s
wasmer-demo-5b7c886887-8lz7t 2/2 Running 0 2m2s
wasmtime-demo-6878c5447-t82gx 2/2 Running 0 2m2s |
MinikubePrerequisites
Setup cluster# Crate cluster
minikube start --container-runtime='containerd'
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached. NAME READY STATUS RESTARTS AGE
lunatic-demo-785446bf5-8vdlc 2/2 Running 0 100s
wasm-slight-9566467b7-nchvc 2/2 Running 0 100s
wasm-spin-8cc64fd85-wwzc5 2/2 Running 0 100s
wasm-wws-ff448f95d-dcwhk 2/2 Running 0 99s
wasmedge-demo-774b4489c5-dcfn7 2/2 Running 0 100s
wasmer-demo-5b7c886887-j6qlv 2/2 Running 0 100s
wasmtime-demo-6878c5447-b9tj7 2/2 Running 0 100s |
MicroK8sPrerequisites
Setup cluster# Crate cluster
# Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-1034-raspi aarch64)
sudo snap install microk8s --channel=1.28-strict/stable
# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
--set kwasmOperator.autoProvision="true" \
--set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42" Test executionkubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml Test resultsDescription: This was a smoke test, all containers start and have a sidecar container attached. NAME READY STATUS RESTARTS AGE
wasmedge-demo-5ff758d79-xj5fs 2/2 Running 0 2m9s
wasmer-demo-857f947cb7-nc99l 2/2 Running 0 2m9s
lunatic-demo-6475554875-lnc7b 2/2 Running 0 2m8s
wasm-slight-6467bcc5bc-k5l62 2/2 Running 0 2m7s
wasm-spin-74c4cf5c77-qtb9p 2/2 Running 0 2m8s
wasmtime-demo-56c78ddd95-8p9bc 2/2 Running 0 2m8s
wasm-wws-888f6bc4b-7cm27 2/2 Running 0 2m7s |
For the v0.3.0 installer, the runwasi shims are built from the master (commit: ba5ab5ada5a4017622f6d93092345baf9cdfb507). |
This is the kwasm-node-installer v0.3.0 release issue. The PR #42 updates
deislabs/containerd-wasm-shims
to v0.9.0 andcontainerd/runwasi
to the latest master (as soon as the next release is out, we will pin the version to it).The latest shims introduced some changes, so we need to test these shims and the installer on all supported platforms. The test documentation consists of three parts: cluster setup, test setup, and test result. The Kind test can ne used as template. Feel free to test K8s distributions you are familiar with and post the test documentation as a reply to this issue.
The text was updated successfully, but these errors were encountered: