-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Config requiring docker network that supports ipv6 which is not a default setting #9000
Comments
This was added as part of the work to test dualstack in Cluster API. I don't think we need to have dualstack on by default in the KIND setup though - if it's causing issues for folks we should revert. Would you be able to open a PR to remove the networking section from that script? |
/triage accepted |
@killianmuldoon But we have it enabled in our e2e tests right? That means when we run our e2e tests against tilt we have a difference in setup if we just drop the setting? (we could make it configurable via env var and default to the current behavior instead of just dropping the setting) I would assume in general the current setting works if IPv6 is available. |
I hit the same issue and also used the workaround described above, btw. If it's relevant I don't have a functional ipv6 environment locally. |
Sure! I will get started on that now |
That's right, @sophiajwitt will you see if you can add a way to configure this instead of just dropping the setting? |
Yes, will look into that |
On the environment variable, could we define something like TILT_USE_DUAL_STACK which, if set, enables the dual stack configuration? We could then add it to the tilt documentation. I'm guessing anybody working on IPv6 specifically is going to go looking for it in the docs if IPv6 isn't there, so discoverability should be good. |
We could look if there is a way to programmatically understand if ipv6 networking is enabled in docker and it should be enabled in kind? We can always fallback to single stack, but if all the checks pass, we can enable dual stack by default on most systems? |
EDIT: IGNORE THIS! Explanation below. How about something like: enable_dual_stack=0
for i in $(seq 0 4); do
if ping -6 -c1 www.google.com; then
enable_dual_stack=1
echo "Functional IPv6 configuration detected. Enabling dual stack"
break
fi
done
if [ $enable_dual_stack == 0 ]; then
echo "Functional IPv6 configuration not detected."
fi That's going to ping google.com with IPv6 up to 5 times. Should weed out misconfigured IPv6 as well as no IPv6. |
The problem is that due to a local MTU requirement we're pre-creating the kind network (as documented somewhere, I forget where). Specifically: $ docker network create -o "com.docker.network.driver.mtu=1200" kind The network this creates doesn't have IPv6 enabled. If I delete this network and let kind create one then
How about a note in the docs, something like: If there is a pre-existing docker kind network configured without IPv6 support you may see:
The docker kind network MUST be configured with IPv6 support. Note that this does not require a functioning IPv6 environment. Deleting the docker kind network will allow kind to create a new network with IPv6. I suspect our actual problem of how to configure a docker kind network with a custom MTU is somewhat esoteric and not worth documenting in CAPI. |
Defining the needed mtu value in a docker daemon config file (/etc/docker/daemon.json) worked for me. Like so:
Then, running the script like usual produces both the cluster and the kind network. The network supports IPv6 and has the correct mtu. You can make sure by running Hope this helps! |
@sophiajwitt Shall we close this? |
I think we should wait for a PR to fix this issue before closing - otherwise it might be forgotten about. |
Do you want to inspect any pre-existing docker network to see if IPv6 is enabled? So proposed logic:
We would then use this to do some templating in ./hack/kind-install-for-capd.sh. We should log a message if we disable dual stack support. |
I think setting an env variable - and defaulting to having I expect the IPv6 case to be the minority case for end-users. We only introduced this recently, and it was only changed to keep the tilt setup in-line with our upstream Prow setup. There's lots of other differences between these two environments though so I don't know how worthwhile it is to invest in keeping this specific networking config directly the same. |
If we can't safely autodetect that a network is properly configured, we can go the environment variable route, and potentially just run some extra checks (if we can) for additional validation. I agree with @killianmuldoon that in the majority of cases, users don't care about ipv6, especially within kind. |
Fine for me. The idea was basically that we use the same configuration in dev and in Prow to minimize the differences in the environment. Dual stack tests are already hard to debug and even harder if the configuration is different. Of course not an issue if "whoever is debugging" is aware of the differences. |
What steps did you take and what happened?
While following the rapid iterative deployment with tilt tutorial (https://cluster-api.sigs.k8s.io/developer/tilt.html), an issue was run into with the preconfigured cluster script
./hack/kind-install-for-capd.sh
as shown belowWork around used:
editing the /hack/kind-install-for-capd.sh file and removing
from the script
What did you expect to happen?
I expected the preconfigured script to work without editing out the networking section.
Is this a common workaround? Or are there other ways to configure docker to use IPv6 correctly?
Cluster API version
v1.4.4
Kubernetes version
No response
Anything else you would like to add?
No response
Label(s) to be applied
/kind bug
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.
The text was updated successfully, but these errors were encountered: