Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate docker v27 ipv6 changes #3677

Open
BenTheElder opened this issue Jul 1, 2024 · 8 comments
Open

Investigate docker v27 ipv6 changes #3677

BenTheElder opened this issue Jul 1, 2024 · 8 comments
Assignees
Labels
area/provider/docker Issues or PRs related to docker kind/bug Categorizes issue or PR as related to a bug.

Comments

@BenTheElder
Copy link
Member

See:

https://docs.docker.com/engine/release-notes/27.0/#ipv6

https://kubernetes.slack.com/archives/CEKK1KTN2/p1719537867758879

kubernetes/test-infra#32863 (comment)

We need to figure out how we want to handle these in Kubernetes's CI and in kind.

/assign @aojea

@BenTheElder BenTheElder added kind/bug Categorizes issue or PR as related to a bug. area/provider/docker Issues or PRs related to docker labels Jul 1, 2024
@aojea
Copy link
Contributor

aojea commented Jul 1, 2024

Did we see any issues yet?

we have this weird problem here kubernetes-sigs/kube-network-policies#47 (comment) , but I do not know if is related

@BenTheElder
Copy link
Member Author

Did we see any issues yet?

Yes.

You can see that just creating the network now flakes in pull-kind-test job, since we updated to docker v27 in CI earlier: kubernetes/test-infra#32863 (comment)

@BenTheElder
Copy link
Member Author

Example failure: #648 (comment)

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kind/648/pull-kind-test/1807905535295492096

A user also reported issues with their own environment and docker v27 in the slack link above.

@BenTheElder
Copy link
Member Author

=== FAIL: pkg/cluster/internal/providers/docker TestIntegrationEnsureNetworkConcurrent (1.45s)
network_integration_test.go:60: error creating network: command "docker network create -d=bridge -o com.docker.network.bridge.enable_ip_masquerade=true -o com.docker.network.driver.mtu=1500 --ipv6 --subnet fc00:3051:9942:af9f::/64 integration-test-ensure-kind-network" failed with error: exit status 1
network_integration_test.go:63: "Error response from daemon: Failed to Setup IP tables: Unable to enable NAT rule: (iptables failed: ip6tables --wait -t nat -I POSTROUTING -s fc00:3051:9942:af9f::/64 ! -o br-4e53c7863d0d -j MASQUERADE: modprobe: FATAL: Module ip6_tables not found in directory /lib/modules/5.15.0-1054-gke\nip6tables v1.8.9 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)\nPerhaps ip6tables or your kernel needs to be upgraded.\n (exit status 3))\n"
network_integration_test.go:65:

@aojea
Copy link
Contributor

aojea commented Jul 2, 2024

Commented in kubernetes/test-infra#32863 (comment)

@BenTheElder
Copy link
Member Author

BenTheElder commented Jul 2, 2024

working on fix in:
kubernetes/test-infra#32890
kubernetes/test-infra#32891
+ triggered an early re-run of autobump job following those and then merging:
kubernetes/test-infra#32881

@BenTheElder
Copy link
Member Author

OK, I think this is mitigated for Kubernetes CI now, after also kubernetes/test-infra#32895

We should still look into the behavior changes more and probably clean up the dind iptables stuff.

@BenTheElder
Copy link
Member Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/provider/docker Issues or PRs related to docker kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants