CTS - Connectivity Test Server - Ubuntu 20.04LTS, Jenkins, Python, Kuebnetes, docker
It contains all the resources required to build and deploy a Jenkins stand-alone pod in Kubernetes.
It contains the groovy Jenkins job definitions for all the Job used to create the CI/CD pipeline.
Contains CTS service and docker
Deplojment via kubernetes in test, staging and production It also contains some placeholder (as SERVICE_NAME, IMAGE_VERSION) to be replaced during the pipeline execution with actual build data
sudo apt-get update
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
sudo apt install curl
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo groupadd docker
sudo usermod -aG docker $USER
docker run hello-world
sudo systemctl start docker
sudo systemctl enable docker
sudo apt install apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
sudo apt install kubeadm kubelet kubectl kubernetes-cni
sudo swapoff -a
sudo nano /etc/fstab
# Comment this line
/swapfile
#to this
#/swapfile
sudo apt install net-tools
sudo systemctl status kublet # see if its actually running
sudo systemctl stop kubelet # stop it
sudo docker stop $(docker ps -a -q)
sudo docker rm $(docker ps -a -q)
docker container prune
sudo systemctl stop docker
sudo systemctl start docker.service
sudo kubeadm reset -f &&
sudo systemctl stop kubelet &&
sudo systemctl stop docker &&
sudo rm -rf /var/lib/cni/ &&
sudo rm -rf /var/lib/kubelet/* &&
sudo rm -rf /etc/cni/ &&
sudo ifconfig cni0 down &&
sudo ifconfig flannel.1 down &&
sudo ifconfig docker0 down &&
sudo ip link delete cni0 &&
sudo ip link delete flannel.1
sudo rm -rf /var/lib/etcd/*
sudo rm -rf $HOME/.kube
sudo systemctl start docker
sudo systemctl enable docker
sudo systemctl start kubelet
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl cluster-info
kubectl get pods -n kube-system
[https://github.com/coreos/flannel]
Flannel runs a small, single binary agent called flanneld on each host, and is responsible for allocating a subnet lease to each host out of a larger, preconfigured address space. Flannel uses either the Kubernetes API or etcd directly to store the network configuration, the allocated subnets, and any auxiliary data (such as the host's public IP).
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
#Control plane node isolation By default, your cluster will not schedule Pods on the control plane nodes for security reasons.
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
export K8S_MASTER=$(kubectl get nodes -o name | cut -d/ -f2)
echo $K8S_MASTER
kubectl describe node $K8S_MASTER
kubectl taint node $K8S_MASTER node-role.kubernetes.io/master:NoSchedule-
git clone https://github.com/sorli2se/cts.git
kubectl create -f jenkins-build/rbac.yaml
sudo apt install jq
JENKINS_TOKEN=$(kubectl get secrets jenkins -o json|jq -r '.data.token'|base64 -d)
echo $JENKINS_TOKEN
mkdir ~/docker-registry
cd ~/docker-registry
mkdir data
nano docker-compose.yml
version: '3'
services:
registry:
image: registry:2
ports:
- "5000:5000"
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
volumes:
- ./data:/data
docker-compose up
/etc/nginx/nginx.conf
Paste content in line
location / {
# Do not allow connections from docker 1.5 and earlier
# docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
return 404;
}
proxy_pass http://localhost:5000;
proxy_set_header Host $http_host; # required for docker client's sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
}
Run docker
docker-compose up
Restart nginx server
sudo systemctl restart nginx
Open in browser url:
https://your_domain/v2
docker build --build-arg K8S_TOKEN=$JENKINS_TOKEN -t jenkins:docker jenkins-build/.
docker tag jenkins:docker localhost/jenkins:docker
docker push localhost/jenkins:docker
kubectl create -f jenkins-build/deployment.yaml
kubectl create -f jenkins-build/service.yaml
# Save Jenkins pod name in env var
export JENKINS_POD=$(kubectl get po -l name=jenkins -o name | cut -d/ -f2)
echo $JENKINS_POD
# Get the admin password from the logs
kubectl logs -f $JENKINS_POD
# Or from inside the container
kubectl exec $JENKINS_POD -- cat /var/jenkins_home/secrets/initialAdminPassword
kubectl exec --stdin --tty $JENKINS_POD -- /bin/bash
kubectl taint node $K8S_MASTER node-role.kubernetes.io/master:NoSchedule-
1.Open url http://localhost:30001 2.And paste string into textfiled to unlock Jenkins 3.Click Install suggested plugins 4.Create first Admin User
##Checkout project job and tool
# Get into the jenkins pod
kubectl exec -ti $JENKINS_POD -- bash
cd /var/jenkins_home/jobs/
git clone https://github.com/sorli2se/cts-jenkins.git .
java -jar /var/jenkins_home/war/WEB-INF/jenkins-cli.jar \
-auth admin:admin \
-s http://127.0.0.1:8080/ \
install-plugin copyartifact job-dsl pipeline-utility-steps
java -jar jenkins-cli.jar \
-auth admin:admin \
-s http://127.0.0.1:8080/ \
safe-restart
Or manually instal plugins: copyartifact job-dsl pipeline-utility-steps
In browser click Manage Jenikins -> Reload Coniguration from Disk
Go to http://localhost:30001:8080/job/kubernetes-config/configure Source Code Managment > Credetntials > Add and add credentials of git project
Go to http://localhost:30001/job/cts-webserver/configure Source Code Managment > Credetntials > Select130 and add credentials of git project
And go to http://localhost:30001/job/kubernetes-config/ And click build
If everthing is ok is last version deployed. If we wanna deploy custom version go to http://localhost:30001/job/cd-job-cts-webserver/ and click build with parameters and select build of CI.
Go to http://localhost:30001/job/kubernetes-config/configure and click Poll SCM Enter text for example "H/15 * * * *" in field Schedule
Or you can use GitHub hook trigger for GITScm polling
kubectl describe services -n test | grep 'Endpoints:'
Endpoints: 10.244.0.50:8080
curl -i -H GET 'http://10.244.0.50:8080/ping'
HTTP/1.0 200 OK Server: SimpleHTTP/0.6 Python/3.8.5 Date: Sat, 31 Oct 2020 17:58:03 GMT Content-Type: text/html; charset=utf-8
kubectl get pods -n test
kubectl -n test exec --stdin --tty cts-webserver-c99dc6785-xx2cr -- /bin/bash
kubectl -n test exec cts-webserver-c99dc6785-xx2cr -- touch /opt/app/do_maintance_mode
curl -i -H GET 'http://10.244.0.50:8080/
curl -i -H GET 'http://10.244.0.50:8080/ping'
HTTP/1.0 503 Service Unavalible Server: SimpleHTTP/0.6 Python/3.8.5 Date: Sat, 31 Oct 2020 21:15:08 GMT Content-Type: text/html; charset=utf-8
kubectl -n test exec cts-webserver-c99dc6785-xx2cr -- rm /opt/app/do_maintance_mode
curl -i -H GET 'http://10.244.0.50:8080/ping'
HTTP/1.0 200 OK Server: SimpleHTTP/0.6 Python/3.8.5 Date: Sat, 31 Oct 2020 21:16:03 GMT Content-Type: text/html; charset=utf-8
optionally specify how much of each resource a Container needs If the node where a Pod is running has enough of a resource available, it's possible (and allowed) for a container to use more resource than its request for that resource specifies. However, a container is not allowed to use more than its resource limit. To specify a CPU request for a container, include the resources:requests field in the Container resource manifest.
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
If you get error by command: kubectl --kubeconfig /etc/kubernetes/config create namespace test error: no matches for kind "Namespace" in version "v1"
The problem is that I login in browser http://ip:port of jenkins end point to Jenikins and not to exposed port.
I have solved this in subject Clean existing enviroment and then continue from begining in subject Initialize Kubernetes and repeat till subject Jenkins Build and deploy.
https://github.com/DevOpsPlayground/Hands-on-with-Jenkins-CI-CD-Pipelines-in-Kubernetes https://www.magalix.com/blog/create-a-ci/cd-pipeline-with-kubernetes-and-jenkins https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/ https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ https://www.digitalocean.com/community/tutorials/how-to-set-up-a-private-docker-registry-on-ubuntu-20-04