Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kube-proxy in kind-kind ending in Error (CrashloopBackoff) #18

Closed
garloff opened this issue Aug 17, 2021 · 6 comments
Closed

kube-proxy in kind-kind ending in Error (CrashloopBackoff) #18

garloff opened this issue Aug 17, 2021 · 6 comments
Labels
Container Issues or pull requests relevant for Team 2: Container Infra and Tooling

Comments

@garloff
Copy link
Contributor

garloff commented Aug 17, 2021

It appears to crash trying to write to /proc/sys/net/netfilter/nf_conntrack_max, which fails (permission denied).
@garloff
Copy link
Contributor Author

garloff commented Aug 17, 2021

It appears that the container does not have the rights to change sysctls ...
Setting the limit to the wanted 131072 manually on the VM helps the kube=proxy to work.
Not sure that it helps the overall failure

@garloff
Copy link
Contributor Author

garloff commented Aug 17, 2021

With this setting, the testcluster creation works again.

@curx
Copy link
Member

curx commented Aug 17, 2021 •
edited
Loading

see kubernetes-sigs/kind#2240 and kubernetes/kubernetes#44919 more in detail

@garloff
Copy link
Contributor Author

garloff commented Aug 17, 2021

Thanks, Thorsten!
The error message was good enough that I could guess the solution -- I wish all errrors would be so easy to solve!
I guess the workaround is to just add a sysctl.d setting to the image (or inject it before we deploy kind).
Just adding to the deploy.sh won't work, as you'll need root privs ...

@curx
Copy link
Member

curx commented Aug 17, 2021

We can update the cloud-config to add the needed sysctl settings, to after the bootstrap the kind is ready to use.

garloff added a commit that referenced this issue Aug 18, 2021
@garloff
Inject sysctl changing nf_conntrack_max to 131072.
7d61cba 
This addresses
#18
kubernetes-sigs/kind#2240

Signed-off-by: Kurt Garloff <scs@garloff.de>
@garloff garloff mentioned this issue Aug 18, 2021
Closed
garloff added a commit that referenced this issue Aug 18, 2021
@garloff
Inject sysctl changing nf_conntrack_max to 131072.
01b0dc0 
This addresses
#18
kubernetes-sigs/kind#2240

Signed-off-by: Kurt Garloff <scs@garloff.de>
@garloff garloff mentioned this issue Aug 18, 2021
Merged
garloff added a commit that referenced this issue Aug 18, 2021
@garloff
Fix/conntrack sysctl2 (#20)
d8eee42 
* Inject sysctl changing nf_conntrack_max to 131072.

This addresses
#18
kubernetes-sigs/kind#2240

* Need to load nf_conntrack kmod for the sysctl setting.

* Add nf_conntrack to modules-load.d to ensure sysctl works.

This is required to be reboot safe.

Signed-off-by: Kurt Garloff <scs@garloff.de>
@garloff
Copy link
Contributor Author

garloff commented Aug 22, 2021

Fixed by PR #20.

@garloff garloff closed this as completed Aug 22, 2021
@jschoone jschoone added the Container Issues or pull requests relevant for Team 2: Container Infra and Tooling label Feb 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Container Issues or pull requests relevant for Team 2: Container Infra and Tooling
Projects
Sovereign Cloud Stack
Archived in project
Milestone
No milestone
Development

No branches or pull requests
3 participants
@curx @jschoone @garloff