Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.3.0 Test Matrix #43

Closed
0xE282B0 opened this issue Sep 12, 2023 · 15 comments
Closed

v0.3.0 Test Matrix #43

0xE282B0 opened this issue Sep 12, 2023 · 15 comments
Labels
good first issue Good for newcomers help wanted Extra attention is needed release

Comments

@0xE282B0
Copy link
Member

0xE282B0 commented Sep 12, 2023

This is the kwasm-node-installer v0.3.0 release issue. The PR #42 updates deislabs/containerd-wasm-shims to v0.9.0 and containerd/runwasi to the latest master (as soon as the next release is out, we will pin the version to it).

The latest shims introduced some changes, so we need to test these shims and the installer on all supported platforms. The test documentation consists of three parts: cluster setup, test setup, and test result. The Kind test can ne used as template. Feel free to test K8s distributions you are familiar with and post the test documentation as a reply to this issue.

K8s Distribution WasmEdge Wasmer WasmTime Lunatic Spin Slight WWS
Kind
MicroK8s
Minikube
CoLima
Killercoda
AWS EKS
Azure AKS
Civo K3s
Digital Ocean K8s
GCP GKE
Namespace.so
@0xE282B0 0xE282B0 added good first issue Good for newcomers help wanted Extra attention is needed release labels Sep 12, 2023
@0xE282B0
Copy link
Member Author

0xE282B0 commented Sep 13, 2023

Kind

Recording: https://asciinema.org/a/SvNWIgRo0xdKQEkCj0cyQ293b

Prerequisites

  • Docker
  • Kind
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
kind create cluster --image kindest/node:v1.28.0

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached..
Notes: There is a known issue where the sidecar does not start on the first run after installation. If you delete it, it will stay in Terminating forever, but the new Pod will start and both containers will work fine.

❯ kubectl get pod                                
NAME                             READY   STATUS        RESTARTS   AGE
lunatic-demo-6475554875-zbrwh    2/2     Running       0          73s
wasm-slight-6467bcc5bc-gx29z     2/2     Running       0          73s
wasm-spin-74c4cf5c77-4slq4       2/2     Running       0          73s
wasm-wws-888f6bc4b-qj6rx         2/2     Running       0          73s
wasmedge-demo-5ff758d79-b5v6j    2/2     Running       0          73s
wasmer-demo-857f947cb7-8qx5g     1/2     Terminating   2          73s
wasmer-demo-857f947cb7-xqw4s     2/2     Running       0          12s
wasmtime-demo-56c78ddd95-x6wmc   2/2     Running       0          73s

@0xE282B0
Copy link
Member Author

0xE282B0 commented Sep 13, 2023

Colima

Recording: https://asciinema.org/a/nF5m7fHgdr3Bq8Qdj1scy66Jq

Prerequisites

  • Colima
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
colima start -k -r containerd --kubernetes-version v1.28.1+k3s1

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.
Notes:

$ df -h
Filesystem                Size      Used Available Use% Mounted on
...
/dev/vda                297.4M    297.4M         0 100% /media/vda
tmpfs                   985.5M    985.5M         0 100% /
tmpfs                   394.2M      2.2M    392.0M   1% /run
/dev/disk/by-label/data-volume
                         58.8G      3.8G     51.9G   7% /mnt/data
/dev/disk/by-label/data-volume
                         58.8G      3.8G     51.9G   7% /usr/local
/dev/disk/by-label/data-volume
                         58.8G      3.8G     51.9G   7% /var/lib

Only runwasi shims become ready:

❯ kubectl get pod                                                                                     
NAME                             READY   STATUS             RESTARTS      AGE
wasmer-demo-857f947cb7-lr7rk     2/2     Running            0             2m39s
wasmtime-demo-56c78ddd95-8jsbz   2/2     Running            0             2m39s
wasmedge-demo-5ff758d79-7v4rx    2/2     Running            0             2m39s
wasm-slight-6467bcc5bc-pqt2x     0/2     CrashLoopBackOff   8 (47s ago)   2m39s
wasm-wws-888f6bc4b-gnzz4         0/2     CrashLoopBackOff   8 (43s ago)   2m39s
lunatic-demo-6475554875-wqphh    0/2     CrashLoopBackOff   8 (30s ago)   2m39s
wasm-spin-74c4cf5c77-m5z5x       0/2     CrashLoopBackOff   3 (11s ago)   2m39s

@0xE282B0
Copy link
Member Author

Namespace.so

Recording: https://asciinema.org/a/QnreeaoqsldJWUi6ULsvpykT0

Prerequisites

  • Namespace.so account & nsc CLI
  • Kubectl

Setup cluster

cat > kwasm.json << EOF
{"disks":[{"name":"kwasm","image":"ghcr.io/kwasm/kwasm-node-installer:pr-42"}],"containerd_shims":[{"name":"lunatic","runtime_type":"/disk/kwasm/assets/containerd-shim-lunatic-v1"},{"name":"slight","runtime_type":"/disk/kwasm/assets/containerd-shim-slight-v1"},{"name":"spin","runtime_type":"/disk/kwasm/assets/containerd-shim-spin-v1"},{"name":"wws","runtime_type":"/disk/kwasm/assets/containerd-shim-wws-v1"},{"name":"wasmedge","runtime_type":"/disk/kwasm/assets/containerd-shim-wasmedge-v1"},{"name":"wasmer","runtime_type":"/disk/kwasm/assets/containerd-shim-wasmer-v1"},{"name":"wasmtime","runtime_type":"/disk/kwasm/assets/containerd-shim-wasmtime-v1"}]}
EOF
nsc create --experimental_from kwasm.json
nsc kubeconfig write 1rq3tlt0mfpu6
export KUBECONFIG=~/Library/Caches/ns/tmp/kubeconfig/973245096.yaml

Test execution

Namespace.so clusters can't restart containerd. When containerd is terminated the whole cluster is terminated. But they have a parameter to customize containerd config before startup.

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.
Notes: There is a known issue where the first Pod does not start on the first run after installation. If you delete it, it will stay in Terminating forever, but the new Pod will start and both containers will work fine.

❯ kubectl get pod                            
NAME                             READY   STATUS        RESTARTS      AGE
wasmer-demo-558c75c49f-p7sf6     2/2     Running       0             30s
wasmtime-demo-6979d575-48nq4     2/2     Running       0             30s
lunatic-demo-767757cf9-2x55j     2/2     Running       0             30s
wasmedge-demo-7fb59fffdc-46k4l   2/2     Running       0             30s
wasm-spin-85bfb69795-8q877       2/2     Running       0             30s
wasm-slight-5646fd7cd5-swtft     2/2     Running       1 (26s ago)   30s
wasm-wws-c875b9664-wf84x         0/2     Terminating   0             29s
wasm-wws-c875b9664-8btjt         2/2     Running       0             5s

@Mossaka
Copy link
Contributor

Mossaka commented Sep 22, 2023

There is a known issue where the sidecar does not start on the first run after installation. If you delete it, it will stay in Terminating forever

Would you please link the issue (if the issue was not created, raise an issue to upstream) here for reference?

@Mossaka
Copy link
Contributor

Mossaka commented Sep 22, 2023

Disk space for / is at 100% after copying the shims

Oh no, the symbol stripping work @jprendes did might help to mitigate the disk pressure issue! https://cloud-native.slack.com/archives/C04LTPB6Z0V/p1695201699156729.

@Mossaka
Copy link
Contributor

Mossaka commented Sep 22, 2023

Thank you for conducting tests on the shims on multiple k8s distributions; your findings are invaluable for us maintainers to understand our operational landscape and potential challenges with each release. To automate this process, I suggest implementing integration tests via GitHub Actions within the upstream repositories, runwasi and containerd-wasm-shims. This would serve as a robust safeguard for future releases.

The challenge lies in, for one, writing tests for two repositories and, for two, accommodating the numerous Kubernetes distributions, as you listed a table. Despite these complexities, I'm willing to dedicate some time to the automation effort for both the runwasi and containerd-wasm-shims repositories.

Please note that I won't have much capacity until I come back from my vacation 🏝️

@0xE282B0
Copy link
Member Author

@Mossaka I don't expect you to do anything while you're on vacation, not even read github issues 😉.

@0xE282B0
Copy link
Member Author

0xE282B0 commented Sep 22, 2023

AWS EKS

Prerequisites

  • aws cli
  • eksctl
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
eksctl create cluster --version 1.27

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached..

❯ kubectl get pod  
NAME                             READY   STATUS    RESTARTS   AGE
lunatic-demo-785446bf5-sfhmw     2/2     Running   0          58s
wasm-slight-9566467b7-ntncd      2/2     Running   0          58s
wasm-spin-8cc64fd85-vvr97        2/2     Running   0          58s
wasm-wws-ff448f95d-gz7qk         2/2     Running   0          58s
wasmedge-demo-774b4489c5-z7blc   2/2     Running   0          59s
wasmer-demo-5b7c886887-pt7ds     2/2     Running   0          59s
wasmtime-demo-6878c5447-r46d8    2/2     Running   0          58s

@0xE282B0
Copy link
Member Author

CIVO K3s

Prerequisites

  • civo cli
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
civo kubernetes create kwasm
civo kubernetes config kwasm --save

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached..

❯ kubectl get pod                                                                                                                                             
NAME                             READY   STATUS    RESTARTS   AGE
wasm-slight-5646fd7cd5-q7mgl     2/2     Running   0          3m7s
wasmer-demo-558c75c49f-p9sq7     2/2     Running   0          3m7s
wasm-wws-c875b9664-s9b57         2/2     Running   0          3m7s
wasmedge-demo-7fb59fffdc-jqm2s   2/2     Running   0          3m7s
wasmtime-demo-6979d575-zs2hl     2/2     Running   0          3m7s
lunatic-demo-767757cf9-8r88m     2/2     Running   0          3m7s
wasm-spin-85bfb69795-cr6nv       2/2     Running   0          3m7s

@0xE282B0
Copy link
Member Author

0xE282B0 commented Sep 22, 2023

Colima

Prerequisites

  • Colima
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
colima start -k -r containerd --kubernetes-version v1.28.1+k3s1 --network-address

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.
Notes:

  • croup v1 is now supported
❯ kubectl get pod
NAME                             READY   STATUS    RESTARTS   AGE
wasm-slight-6467bcc5bc-dshjf     2/2     Running   0          2m3s
wasm-spin-74c4cf5c77-zc2pv       2/2     Running   0          2m3s
wasm-wws-888f6bc4b-lj8nh         2/2     Running   0          2m3s
lunatic-demo-6475554875-kv497    2/2     Running   0          2m3s
wasmedge-demo-5ff758d79-b92kt    2/2     Running   0          2m3s
wasmer-demo-857f947cb7-82kpn     2/2     Running   0          2m3s
wasmtime-demo-56c78ddd95-zmqhp   2/2     Running   0          2m3s

@0xE282B0
Copy link
Member Author

DigitalOcean Kubernetes

Prerequisites

  • doctl
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
doctl kubernetes cluster create kwasm
doctl kubernetes cluster kubeconfig save kwasm      

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.

❯ kubectl get pod
NAME                             READY   STATUS    RESTARTS   AGE
lunatic-demo-6475554875-fxrr4    2/2     Running   0          49s
wasm-slight-6467bcc5bc-jbkkg     2/2     Running   0          49s
wasm-spin-74c4cf5c77-22zkt       2/2     Running   0          49s
wasm-wws-888f6bc4b-xd8k5         2/2     Running   0          48s
wasmedge-demo-5ff758d79-r6h9m    2/2     Running   0          49s
wasmer-demo-857f947cb7-rsrzz     2/2     Running   0          49s
wasmtime-demo-56c78ddd95-j4cwb   2/2     Running   0          49s

@0xE282B0
Copy link
Member Author

Killercoda.com

Prerequisites

  • browser

Setup cluster

visit: https://killercoda.com/playgrounds/scenario/kubernetes

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.

NAME                             READY   STATUS    RESTARTS   AGE
lunatic-demo-785446bf5-zjg2w     2/2     Running   0          2m2s
wasm-slight-9566467b7-jcgjd      2/2     Running   0          2m2s
wasm-spin-8cc64fd85-zc5d2        2/2     Running   0          2m2s
wasm-wws-ff448f95d-lpx4n         2/2     Running   0          2m2s
wasmedge-demo-774b4489c5-bgdwf   2/2     Running   0          2m2s
wasmer-demo-5b7c886887-8lz7t     2/2     Running   0          2m2s
wasmtime-demo-6878c5447-t82gx    2/2     Running   0          2m2s

@0xE282B0
Copy link
Member Author

Minikube

Prerequisites

  • Docker
  • minikube
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
minikube start --container-runtime='containerd'

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.

NAME                             READY   STATUS    RESTARTS   AGE
lunatic-demo-785446bf5-8vdlc     2/2     Running   0          100s
wasm-slight-9566467b7-nchvc      2/2     Running   0          100s
wasm-spin-8cc64fd85-wwzc5        2/2     Running   0          100s
wasm-wws-ff448f95d-dcwhk         2/2     Running   0          99s
wasmedge-demo-774b4489c5-dcfn7   2/2     Running   0          100s
wasmer-demo-5b7c886887-j6qlv     2/2     Running   0          100s
wasmtime-demo-6878c5447-b9tj7    2/2     Running   0          100s

@0xE282B0
Copy link
Member Author

MicroK8s

Prerequisites

  • microk8s
  • Kubectl
  • Helm

Setup cluster

# Crate cluster
# Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-1034-raspi aarch64)
sudo snap install microk8s --channel=1.28-strict/stable

# Install with 0.3.0-pre installer
helm repo add kwasm http://kwasm.sh/kwasm-operator/ --force-update
helm repo update
helm upgrade --install -n kwasm --create-namespace kwasm-operator kwasm/kwasm-operator \
  --set kwasmOperator.autoProvision="true" \
  --set kwasmOperator.installerImage="ghcr.io/kwasm/kwasm-node-installer:pr-42"

Test execution

kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/runtimeclass.yaml
kubectl apply -f https://gist.githubusercontent.com/0xE282B0/1854ad6ed7ddaf4ad7d3032397056a38/raw/937c56555532d8f7dadedc9405c6764f3133073e/workload.yaml

Test results

Description: This was a smoke test, all containers start and have a sidecar container attached.

NAME                             READY   STATUS    RESTARTS   AGE
wasmedge-demo-5ff758d79-xj5fs    2/2     Running   0          2m9s
wasmer-demo-857f947cb7-nc99l     2/2     Running   0          2m9s
lunatic-demo-6475554875-lnc7b    2/2     Running   0          2m8s
wasm-slight-6467bcc5bc-k5l62     2/2     Running   0          2m7s
wasm-spin-74c4cf5c77-qtb9p       2/2     Running   0          2m8s
wasmtime-demo-56c78ddd95-8p9bc   2/2     Running   0          2m8s
wasm-wws-888f6bc4b-7cm27         2/2     Running   0          2m7s

@0xE282B0
Copy link
Member Author

For the v0.3.0 installer, the runwasi shims are built from the master (commit: ba5ab5ada5a4017622f6d93092345baf9cdfb507).
With this test I'm confident to release the installer. When runwasi is released, I will create a patch release to use the same artifacts that are being released.

Thank you @jprendes and @Mossaka!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed release
Projects
None yet
Development

No branches or pull requests

2 participants