In life, video and log data are very large, and in order to effectively preserve and manage them, the environment of edge, fog, and cloud computing must be included. Therefore, the goal of this environment is to develop non-reversive storage management technologies based on blockchain and ipfs in a scalable cloud computing environment.
This is an example of kubernetes between ipfs nodes. In this example, there are three nodes:
- master node (mac-mini)
- worker node1 (mac-mini)
- worker node2 (pi)
Nodes with name worker node1 and worker node2 running as a child process in kubernetes. Master node handle to the go-ipfs that start daemon with pub-sub-experiment flag using CLI command. When you want docker swarm, please click private-network folder.
- ubuntu LTS 18.04
- kubernetes 1.13
- docker 18.09.5
- go 1.12
- go-ipfs 0.4.19
Sharing url: https://github.com/alexellis/k8s-on-raspbian
Static network ip for mac mini and raspberry pi: https://altongmon.tistory.com/637, https://hiseon.me/2018/09/06/ubuntu-18-04-netplan/
- Docker Install and update
curl -sSL get.docker.com | sh && \
sudo usermod -aG docker $USER \
sudo newgrp docker \
sudo apt-get update && sudo apt-get upgrade -y
If it doesn't work, we can use this:
sudo usermod NAME -aG docker
sudo reboot
Disable Swap and * Edit /boot/cmdline.txt
sudo swapoff -a
sudo dphys-swapfile swapoff && \
sudo dphys-swapfile uninstall && \
sudo update-rc.d dphys-swapfile remove
echo Adding "cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory" to /boot/cmdline.txt
-
Now reboot!! do not skip this step.
-
Add repo list and install
kubeadm
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \
sudo apt-get update -q && \
sudo apt-get install -qy kubeadm
-
You now have two new commands installed:
-
kubeadm - used to create new clusters or join an existing one
-
kubectl - the CLI administration tool for Kubernetes
-
Pre-pull images
kubeadm
now has a command to pre-pull the requisites Docker images needed to run a Kubernetes master, type in:
sudo kubeadm config images pull -v3
If using Flannel:
- Initialize your master node with a Pod network CIDR:
sudo kubeadm init --token-ttl=0 --pod-network-cidr=10.43.0.0/16
If using Weave Net
- Initialize your master node:
sudo kubeadm init --token-ttl=0
After the init
is complete run the snippet given to you on the command-line:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Your join token is valid for 24 hours, so save it into a text file. Here's an example of mine:
sudo kubeadm join 192.168.1.49:6443 --token 23oulg.on7eyrbnetdyolp8 --discovery-token-ca-cert-hash sha256:6a583e6385e895971dcc9dc78505e14c1ba6f429f66aee53ab1ac3d4abac733d
If you do not remember the join, check the kubeadm join history again ([preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ error execution phase preflight: couldn't validate the identity of the API Server: expected a 32 byte SHA-256 hash, found 29 bytes):
sudo kubeadm token create --print-join-command
When a master node uses $ kubectl get nodes
, we see Not Ready
. By default, your cluster will not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, run:
kubectl taint nodes --all node-role.kubernetes.io/master-
Install flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
If not install flannel
kubectl -n kube-system apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml
On each node that joins including the master:
sudo sysctl net.bridge.bridge-nf-call-iptables=1
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
23ipfscluster Ready <none> 25d v1.14.1
wrl-01macmini Ready master 36d v1.13.4
wrl-02macmini Ready <none> 25d v1.14.1
Sharing url: https://docs.aws.amazon.com/ko_kr/eks/latest/userguide/dashboard-tutorial.html
Containers packaged with all dependencies and software for IoT applications are portable, light and secure. Containerized IoT applications/services deployed using Docker containers:
sudo docker run -d -it --name docker-ipfs -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4002:4002 -p 127.0.0.1:8081:8081 -p 127.0.0.1:5002:5002 shapna/go-ipfs_node1:Node1
Developled IoT services with P2P pubsub communication model based on Interplanetary File System (IPFS):
sudo docker run -d -it --name docker-ipfs -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4003:4003 -p 127.0.0.1:8082:8082 -p 127.0.0.1:5003:5003 shapna/go-ipfs_node1:Node1 --enable-pubsub-experiment
containerized ipfs in raspberry pi 3b:
sudo docker run -d --name ipfs_host -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 yrzr/go-ipfs-arm32v7:latest --enable-pubsub-experiment
Sharing arm32v7-ipfs url: https://hub.docker.com/r/yrzr/go-ipfs-arm32v7/
A platform to orchestrate all these containers along with their varying workloads, computing, networking, and storage are in demand. And their Orchestration managed by Kubernetes. dockerized ipfs in raspberry pi using kubectl (+nodeselector):
kubectl run --generator=run-pod/v1 pi-ipfs3 --image=yrzr/go-ipfs-arm32v7:latest --overrides='{ "apiVersion": "v1"O, "spec": { "nodeSelector": { "worker-node": "pi" } } }' --port 8087 -- daemon --enable-pubsub-experiment
dockerized ipfs in mac mini using kubectl (+nodeselector):
kubectl run --generator=run-pod/v1 mac2-ipfs1 --image=shapna/go-ipfs_node1:Node1 --overrides='{ "apiVersion": "v1", "spec": { "nodeSelector": { "worker-node": "mac" } } }' --port 8083 -- daemon --enable-pubsub-experiment
The connection to the server 192.168.1.48:6443 was refused - did you specify the right host or port? or after reboot kubectl command not work:
cd ~/.kube
rm -rf config
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=$HOME/.kube/config
sudo systemctl restart kubelet.service
sudo systemctl restart docker
sudo systemctl restart kubelet
ubuntu cgroup error:
sudo nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1"
sudo grub-update && sudo reboot
The connection to the server localhost:8080 was refused – did you specify the right host or port? or Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes") To fix this issue, simply export the admin.conf on the master node:
sudo cp /etc/kubernetes/admin.conf $HOME
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
if not kubeadm join, kubernetes clean delete:
systemctl stop kubelet \
systemctl stop docker \
kubeadm reset
rm -rf /var/lib/cni/
rm -rf /var/lib/kubelet/*
rm -rf /run/flannel
rm -rf /etc/cni/
rm -rf /etc/kubernetes
rm -rf /var/lib/etcd/
rm -rf ~/.kube
ip link delete cni0
ip link delete flannel.1
And then install, first uses $ systemctl start docker
kube-system kube-flannel-ds-arm-7p5pl CrashLoopBackOff
error:
kubectl delete pods deployment --namespace=kube-system kube-flannel-ds-arm-7p5pl
sudo reboot
heapster, monitoring-influxdb error and give label for worker node:
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl label node 25ipfscluster node-role.kubernetes.io/worker=worker
Shutdown kubectl proxy:
killall kubectl
Search for and install the openssh-server package from Ubuntu Software Center. Or run command below in console if you’re on Ubuntu Server without GUI:
sudo apt-get install openssh-server
To edit settings - To change the port 22, root login permission, you may edit the /etc/ssh/sshd_config
file via:
sudo nano /etc/ssh/sshd_config
After you changed the configurations, press Ctrl+X followed by typing y and hitting enter to save the file.
Finally restart the SSH service to apply the changes:
sudo service ssh restart
if not, host connection this:
ssh-keygen -R 192.168.1.49
first, we have to check node label:
kubectl get nodes --show-labels
label create:
kubectl label nodes wrl-02macmini worker-node=mac
label delete:
kubectl label nodes wrl-02macmini worker_node-
kubectl select node and label for nginx:
kubectl run --generator=run-pod/v1 nginx2 --image=nginx --overrides='{ "apiVersion": "v1", "spec": { "nodeSelector": { "workernode": "1" } } }'
kubectl select node and label:
kubectl run --generator=run-pod/v1 pi-ipfs --image=shapna/go-ipfs_node1:Node1 --overrides='{ "apiVersion": "v1", "spec": { "nodeSelector": { "workernode": "1" } } }' --port 8083 -- daemon --enable-pubsub-experiment