Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modifying kubelet to use config files instead of kubelet flags which are about to deprecate. #90

Merged
merged 6 commits into from
Nov 13, 2018
Merged

Modifying kubelet to use config files instead of kubelet flags which are about to deprecate. #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1855303
change Kubelet to use config file
Nov 1, 2018
f1d04a1
changing install-worker.sh to support kubelet config file
Nov 1, 2018
0e22532
included JQ to modify cconfig-kubelet.conf instead of sed
Nov 7, 2018
3fcc6c7
Modified install-worker.sh
Nov 7, 2018
9d355f9
renaming config-kubelet.conf to kubelet-config.json
Nov 7, 2018
b2bde50
Merge branch 'master' into master
micahhausler Nov 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions files/bootstrap.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,21 +105,19 @@ DNS_CLUSTER_IP=10.100.0.10
if [[ $INTERNAL_IP == 10.* ]] ; then
DNS_CLUSTER_IP=172.20.0.10;
fi
echo "$(jq .clusterDNS=[\"$DNS_CLUSTER_IP\"] kubelet-config.json)" > kubelet-config.json

if [[ "$USE_MAX_PODS" = "true" ]]; then
MAX_PODS_FILE="/etc/eks/eni-max-pods.txt"
MAX_PODS=$(grep $INSTANCE_TYPE $MAX_PODS_FILE | awk '{print $2}')
if [[ -n "$MAX_PODS" ]]; then
cat <<EOF > /etc/systemd/system/kubelet.service.d/20-max-pods.conf
[Service]
Environment='KUBELET_MAX_PODS=--max-pods=$MAX_PODS'
EOF
echo "$(jq .maxPods=$MAX_PODS kubelet-config.json)" > kubelet-config.json
fi
fi

cat <<EOF > /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf
[Service]
Environment='KUBELET_ARGS=--node-ip=$INTERNAL_IP --cluster-dns=$DNS_CLUSTER_IP --pod-infra-container-image=602401143452.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/eks/pause-amd64:3.1'
Environment='KUBELET_ARGS=--node-ip=$INTERNAL_IP --pod-infra-container-image=602401143452.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/eks/pause-amd64:3.1'
EOF

if [[ -n "$KUBELET_EXTRA_ARGS" ]]; then
Expand Down
29 changes: 29 additions & 0 deletions files/kubelet-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"kind": "KubeletConfiguration",
"apiVersion": "kubelet.config.k8s.io/v1beta1",
"address": "0.0.0.0",
"authentication": {
"anonymous": {
"enabled": false
},
"webhook": {
"cacheTTL": "2m0s",
"enabled": true
},
"x509": {
"clientCAFile": "/etc/kubernetes/pki/ca.crt"
}
},
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
"clusterDomain": "cluster.local",
"cgroupDriver": "cgroupfs",
"featureGates": {
"RotateKubeletServerCertificate": true
}
}
26 changes: 9 additions & 17 deletions files/kubelet.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,26 +5,18 @@ After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/bin/kubelet \
--address=0.0.0.0 \
--authentication-token-webhook \
--authorization-mode=Webhook \
--allow-privileged=true \
--cloud-provider=aws \
--cluster-domain=cluster.local \
--cni-bin-dir=/opt/cni/bin \
--cni-conf-dir=/etc/cni/net.d \
--container-runtime=docker \
--network-plugin=cni \
--cgroup-driver=cgroupfs \
--register-node=true \
--kubeconfig=/var/lib/kubelet/kubeconfig \
--feature-gates=RotateKubeletServerCertificate=true \
--anonymous-auth=false \
--client-ca-file=/etc/kubernetes/pki/ca.crt $KUBELET_ARGS $KUBELET_MAX_PODS $KUBELET_EXTRA_ARGS
ExecStartPre=/sbin/iptables -P FORWARD ACCEPT
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this gets added here?

Copy link
Contributor Author

@nithu0115 nithu0115 Dec 19, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@errordeveloper - I saw many miss this iptables rule causing problem, hence I added it to ExecStartPre to enforce this command run every time kubelet starts and change the rule.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it part of this PR and there was no discussion about it here? Maybe a comment in the unit file would help also, but I still don't quite understand the purpose, it seems very ad-hoc and as far as I can tell redundant in the context of this AMI.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the above questions is addressed some where else?
why is the FORWARD rule to ACCEPT?

ExecStart=/usr/bin/kubelet --cloud-provider aws \
--config /etc/systemd/system/kubelet.service.d/kubelet-config.json \
--kubeconfig /var/lib/kubelet/kubeconfig \
--allow-privileged true\
--container-runtime docker \
--network-plugin cni \
--register-node true $KUBELET_ARGS $KUBELET_EXTRA_ARGS

Restart=on-failure
RestartSec=5
KillMode=process

[Install]
WantedBy=multi-user.target
5 changes: 4 additions & 1 deletion install-worker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ sudo yum install -y \
aws-cfn-bootstrap \
conntrack \
curl \
jq \
nfs-utils \
ntp \
socat \
Expand All @@ -37,7 +38,6 @@ sudo pip install --upgrade awscli
################################################################################

# Enable forwarding via iptables
sudo iptables -P FORWARD ACCEPT
sudo bash -c "/sbin/iptables-save > /etc/sysconfig/iptables"

sudo mv $TEMPLATE_DIR/iptables-restore.service /etc/systemd/system/iptables-restore.service
Expand Down Expand Up @@ -112,12 +112,15 @@ for binary in ${BINARIES[*]} ; do
done
sudo rm *.sha256

sudo mkdir -p /etc/systemd/system/kubelet.service.d
sudo mv $TEMPLATE_DIR/kubelet-kubeconfig /var/lib/kubelet/kubeconfig
sudo chown root:root /var/lib/kubelet/kubeconfig
sudo mv $TEMPLATE_DIR/kubelet.service /etc/systemd/system/kubelet.service
sudo mv $TEMPLATE_DIR/kubelet-config.json /etc/systemd/system/kubelet.service.d/kubelet-config.json
sudo chown root:root /etc/systemd/system/kubelet.service
sudo mkdir -p /etc/systemd/system/kubelet.service.d


sudo systemctl daemon-reload
# Disable the kubelet until the proper dropins have been configured
sudo systemctl disable kubelet
Expand Down