Feature: Refactoring Sealos to Support k3s, k0s Runtime

[cubxxw]

Problem Description

Sealos currently does not support the k3s runtime and requires the redesign of the runtime interface to avoid excessive coupling with the Processor and to add verification.

Here is a record of testing various k3s installation methods. If you're curious about k3s, be sure to read this note, as well as k3s • official and documentation, and its GitHub • open source address.

Solution

Design a new runtime interface

Redesign the runtime interface to avoid excessive coupling with the Processor and add verification.

Design ideas for the k3s runtime module

To support the k3s runtime, we need to add a new runtime module. This module needs to support the following functions:

• Execute k3s installation commands
• Execute k3s upgrade commands
• Execute k3s uninstall commands

Design the k3s directory and filesystem distribution using Sealos' image distribution logic

To support the k3s runtime, we need to use Sealos' image distribution logic to design the k3s directory and filesystem distribution of the k3s image.

Optimize the packaging of kubeadm and k3s binaries

To support the k3s runtime, we need to optimize the packaging of kubeadm and k3s binaries.

Provide unit test and CI flow to ensure code quality

To ensure code quality, we need to provide unit testing and CI flow.

Tasks

Type	Improvements	Issue	PR	Status
🚀 Feature	Design a new runtime interface to avoid excessive coupling with the Processor and add verification	#1943, #2037	#3741,#3691	✅
🔨 Enhancement	Implement the k3s runtime module to support k3s installation, upgrade, and uninstall commands			❌
🔧 Maintenance	Design the k3s directory and filesystem distribution using Sealos' image distribution logic			❌
👌 Refactoring	Optimize the packaging of kubeadm and k3s binaries			❌
🧪 Testing	Provide unit testing and CI flow to ensure code quality			❌
📈 Metrics	Compare k3s and k0s, as well as Kubernetes for various changes in sealos, for users to choose according to their scenarios			❌
🎯 Fix	The current logic of sealos run will parse the environment variable $, which inevitably needs to be fixed.	#2861	#2862	✅

[stale]

This issue has been automatically closed because we haven't heard back for more than 60 days, please reopen this issue if necessary.

[cuisongliu]

repen

[stale]

This issue has been automatically closed because we haven't heard back for more than 60 days, please reopen this issue if necessary.

[cuisongliu]

./k3s-arm64 server --container-runtime-endpoint  /run/containerd/containerd.sock  --cluster-init --tls-san=192.168.64.189  --kubelet-arg=image-service-endpoint=unix:///var/run/image-cri-shim.sock  --kubelet-arg=pod-infra-container-image=sealos.hub:5000/rancher/mirrored-pause:3.6  --kubelet-arg=cgroup-driver=systemd

1. containerd and kubelet pause image is same
2. containerd and kubelet cgroup-driver is same
3. add image-service-endpoint to kubelet flags

[cuisongliu]

type serviceType int

const (
	openRC serviceType = iota
	systemd
)

const systemdTpl = `
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target
After=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-{{ .envFile }}
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=/bin/sh -xc '! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service'
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart={{ .binDir }}/k3s {{ .exec }}
`

const openRCTpl = `
#!/sbin/openrc-run

LOG_FILE=/var/log/k3s.log

depend() {
    after network-online
    want cgroups
}

start_pre() {
    rm -f /tmp/k3s.*
}

supervisor=supervise-daemon
name=k3s
command="{{ .binDir }}/k3s"
command_args={{ .exec }}

output_log=${LOG_FILE}
error_log=${LOG_FILE}

pidfile="/var/run/k3s.pid"
respawn_delay=5
respawn_max=0

set -o allexport
if [ -f /etc/environment ]; then . /etc/environment; fi
if [ -f {{ .envFile }} ]; then . {{ .envFile }}; fi
set +o allexport
`

[cuisongliu]

using sealos v4.4.0+

[sealos-ci-robot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

using sealos v4.4.0+