add KubeEdge installation procedure.

[fujitatomoya]

part of #3

[fujitatomoya]

keadm init is failing with timeout, i guess that kubernetes version is incompatible with kubeedge. assumption is kubeedge requires docker running at the initialization.

• https://qiita.com/pideoh/items/cac19a32fdc723cca68d (example process)

and after edgecore is running, it can set it back to containerd CRI only w/o docker. which is not really good procedure since it requires docker anyway, that means edge devices need to install the docker anyway...

see https://kubeedge.io/docs/setup/prerequisites/runtime

[fujitatomoya]

TODO

• Successfully deploy containers using KubeEdge, but ROS 2 talker and listener does not communicate each other. probably flannel is used on kubernetes side but edgecore uses host NIC only, discovery does not work.
• https://release-1-14.docs.kubeedge.io/docs/setup/install-with-keadm#enable-kubectl-logs-feature needs to be supported with another issue.
• https://github.com/kubeedge/website/pull/272/files is really useful debug information. find out the original site to link.

[fujitatomoya]

Still having the problem, talker and listener cannot communicate with each other.

root@tomoyafujita:/home/tomoyafujita/DVT/github.com/fujitatomoya/ros_k8s/yaml# kubectl get pods -o wide
NAME                               READY   STATUS    RESTARTS   AGE     IP              NODE           NOMINATED NODE   READINESS GATES
ros2-listener-1-869cd7dc99-5pdbd   1/1     Running   0          7m20s   192.168.1.209   tomoyafujita   <none>           <none>
ros2-talker-1-bdd899d8d-27gnk      1/1     Running   0          7m20s   192.168.1.238   ubuntu         <none>           <none>
root@tomoyafujita:/home/tomoyafujita/DVT/github.com/fujitatomoya/ros_k8s/yaml# kubectl get nodes -o wide
NAME           STATUS   ROLES                  AGE   VERSION                    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
tomoyafujita   Ready    control-plane,master   45m   v1.23.17                   192.168.1.209   <none>        Ubuntu 22.04.3 LTS   6.2.0-33-generic    docker://24.0.6
ubuntu         Ready    agent,edge             38m   v1.22.6-kubeedge-v1.12.1   192.168.1.238   <none>        Ubuntu 22.04.3 LTS   5.15.0-1034-raspi   docker://24.0.5

deployment seems to be as expected,

tomoyafujita@~/DVT/github.com/fujitatomoya/ros_k8s >docker exec -it 238fa3d2e823 /bin/bash
root@tomoyafujita:/# ifconfig -a
...
enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.209  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 2600:1700:1151:83b0:a7e6:f04b:b127:683a  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::fd49:9d5c:dddc:13ea  prefixlen 64  scopeid 0x20<link>
        inet6 2600:1700:1151:83b0::45  prefixlen 128  scopeid 0x0<global>
        inet6 2600:1700:1151:83b0:ab65:5d2f:158a:3515  prefixlen 64  scopeid 0x0<global>
        ether b0:7b:25:17:ab:e4  txqueuelen 1000  (Ethernet)
        RX packets 1488102  bytes 2021436963 (2.0 GB)
        RX errors 0  dropped 54467  overruns 0  frame 0
        TX packets 271953  bytes 43561665 (43.5 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
...
weave: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1376
        inet 10.32.0.1  netmask 255.240.0.0  broadcast 10.47.255.255
        inet6 fe80::8cd5:96ff:fea0:de4e  prefixlen 64  scopeid 0x20<link>
        ether 8e:d5:96:a0:de:4e  txqueuelen 1000  (Ethernet)
        RX packets 6344  bytes 476220 (476.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8574  bytes 1638974 (1.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

we can still see the weavenet interface which has been already removed from the cluster...

I probably need to remove the CNI binaries to be called by CRI.

rm -f /opt/cni/bin/weave*
rm -rf /etc/cni/net.d
ip link delete weave

still not working, seems like localhost communication only.

checking with docker, seems that multicat is not working in my local environment...

[fujitatomoya]

okay, multicast is not working with my local environment now. i need to set up the another environment.

tomoyafujita@~/DVT/github.com/fujitatomoya/ros_k8s >iperf -s -u -B 226.94.1.1 -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Joining multicast group  226.94.1.1
Server set to single client traffic mode (per multicast receive)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------

ubuntu@ubuntu:~$ iperf -c 226.94.1.1 -u -T 32 -t 3 -i 1
------------------------------------------------------------
Client connecting to 226.94.1.1, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11215.21 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 192.168.1.238 port 60908 connected with 226.94.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-1.0000 sec   131 KBytes  1.07 Mbits/sec
[  1] 1.0000-2.0000 sec   128 KBytes  1.05 Mbits/sec
[  1] 2.0000-3.0000 sec   128 KBytes  1.05 Mbits/sec
[  1] 0.0000-3.0171 sec   389 KBytes  1.06 Mbits/sec
[  1] Sent 272 datagrams

no response on server's terminal...

[fujitatomoya]

if that network environment does not work multicast, this cannot be verified. it would be better to add documentation for that. but overall good.