Multi host ignite VMs networking based on WeaveNet not working as expected (reopening issue #628) #642
Comments
|
Hi @mdundek 👋 Your Weave Net setup looks correct, but it seems like the Ignite VMs are not attaching to it at all. The Have a look at what you have in For reference, this is what |
twelho
added
area/networking
|
Hello twelho, thanks for your quick response, I started from 2 fresh VMs to test this again, but this time I started the WeaveNet CNI first before I installed Ignite and the Ignite default CNI plugin. This time I do have the file
I can ping Thanks |
|
Could you post the Docker logs for the weave-kube containers on both hosts, so the output of |
|
IP of host one is: Host 1 & host 2 can talk to each other, host 1 can talk to VM 1 (running on host 1), but host 1 can not talk to VM 2 (running on host 2). Thank you twelho for helping me out on this, much appreciated ! |
|
Yes, from your log output it seems like the two Weave Net instances are able to connect to each other. I just replicated the same setup with two physical hosts on the same LAN, just to end up at the same conclusion – the Weave Net instances pair just fine and the VMs connect to their respective instances, but no traffic flows in the CNI network. The VMs reach their own CNI provider but cannot connnect to the other host nor the internet. This is kind of a dead end now with Weave Net not being intended to be run "externally" like this (would need more work to investigate why it doesn't work), however I'm assembling some scripts to set up a Flannel-based CNI overlay with the aim to get multi-node networking actually working now. Should have the scripts ready for testing tomorrow. |
|
Thanks twelho, appreciate your efforts in helping me out on this. I am very much looking forward to your Flannel based solution, I believe it makes a lot of sense to get multi-host networking work with Ignite, especially when using Ignite with Kubernetes where each node should be running on it's own host when not using it purely for testing purposes. Looking forward to your scripts, thanks again! |
|
The scripts (and docs) are ready in #645, go ahead and give them a go, it would be good to know if they work on your end as well. Thanks for bringing up the static IPs, I've added a chapter to the docs in that same PR, the tl;dr of which being that it's possible but you need to configure it for your CNI provider. For Flannel it might be doable using leases and reservations, but I don't have the resources to implement full-blown CLI utilities to do that right now. The current tools are quite simple since they mostly serve as examples in their current state, but contributions are definitely welcome if there is a need for more advanced configuration. |
|
Helo twelho, thanks, I tested your scrips, it works great! I do have a couple of comments for you:
Thats it. Great work, and thanks again for the support! Very much appreciated. When will you merge your changes to the master branch? BR |
|
Hi,
The merge of #645 will most likely happen early next week. Good luck with your project, just note that we don't have a lot of free resources to put into developing/supporting Ignite currently, so there might be delays in support/bug fixing. |
Hello WeaveWorks team,
I am reopening issue #628, the applied fix did make things slightly better but it did not fix the underlying issue.
Two Hosts, on both I run the WeaveWorks CNI docker image as described in issue #628.
I then installed
igniteon both hosts and started a Ignite VM on each host with the flag--network-plugin cnirespectively. On each host, the output ofifconfigis (I filtered out the other network interfaces that are not relevant to this issue):and
I can ping
10.40.0.0from10.32.0.1and vise versa, this tells me that WeaveWorks CNI is working as expected.But when I SSH into each VM, they both have the same output for
ifconfig:As you can see, they both have the same IP on the WeaveWorks CNI network (obviously not what was expected), rather than each having a dedicated IP. There is no more hanging or delay, so that issue is gone now, but I still cant do multi host Ignite VM networking using WeaveNet CNI.
I also removed the file
/etc/cni/net.d/10-ignite.conflistbefore running theignite runcommand. The content of the newly created/etc/cni/net.d/10-ignite.conflistfile is:{ "cniVersion": "0.4.0", "name": "ignite-cni-bridge", "plugins": [ { "type": "bridge", "bridge": "ignite0", "isGateway": true, "isDefaultGateway": true, "promiscMode": true, "ipMasq": true, "ipam": { "type": "host-local", "subnet": "10.61.0.0/16" } }, { "type": "portmap", "capabilities": { "portMappings": true } }, { "type": "firewall" } ] }Or the documentation is lacking some instructions here, or there is a bug somewhere. Could someone please have a look and help me solve this issue please?
Thanks
The text was updated successfully, but these errors were encountered: