manifest: Add NetworkManager-team and teamd

[cgwalters]

http://bugzilla.redhat.com/1758162

teamd is needed for teaming at all and NetworkManager-team
is needed if we want to drop down files on the
host for teaming and have NetworkManager bring them up.

The argument for adding this to the host by default is
that particularly on some bare metal installs, one wants to use
teaming for the "primary" network interface, which one would
then use to e.g. pull containers. There's already code in
dracut which handles setting up teaming on the kernel cmdline too.

[jlebon]

But do you actually need teaming to bring up networking at all on first boot though? Presumably you'd be teaming interfaces which already have connectivity, right? IOW, why does this not fall under coreos/fedora-coreos-tracker#401?

Also worth printing the final diff and sizes (this made me think of #176 -- updated that!).

[cgwalters]

But do you actually need teaming to bring up networking at all on first boot though?

It's true that a valid path is doing Ignition provisioning with one NIC active and only switching to teaming for subsequent boots.

IOW, why does this not fall under coreos/fedora-coreos-tracker#401?

The argument in the commit message is again that basically one needs networking to fetch container images (or rpm-md repos of extensions) and that dracut already has support for configuring teaming. In contrast usbguard e.g. isn't in any way near that critical path.

[cgwalters]

Additions are:

  NetworkManager-team-1:1.20.10-1.fc31.x86_64
  libdaemon-0.14-18.fc31.x86_64
  libnl3-cli-3.5.0-1.fc31.x86_64
  libteam-1.29-2.fc31.x86_64
  teamd-1.29-2.fc31.x86_64

[dustymabe]

But do you actually need teaming to bring up networking at all on first boot though?

It's true that a valid path is doing Ignition provisioning with one NIC active and only switching to teaming for subsequent boots.

IOW, why does this not fall under coreos/fedora-coreos-tracker#401?

The argument in the commit message is again that basically one needs networking to fetch container images (or rpm-md repos of extensions)

Seems like a good argument to me.

and that dracut already has support for configuring teaming.

I will admit in my testing over the past few days I've found the dracut support to have some question marks. It seems to bring up teaming in the initramfs fine, but when switching to the real root the teamd process that was started in the initrmafs to support the interfaces was gone. My suggestion was going to be that users start with a single NIC and then lay down networking configuration that would then be brought up by NM. Hopefully with NM in the initramfs the problem I saw won't be an issue.

In contrast usbguard e.g. isn't in any way near that critical path.

agreed

[dustymabe]

Assuming we get agreement on inclusion the code changes LGTM.

[jlebon]

The argument in the commit message is again that basically one needs networking to fetch container images (or rpm-md repos of extensions)

Seems like a good argument to me.

OK, let me see if I can expand on this a bit more. So we agree teaming is not required on the first boot. But e.g. in the case of OCP/OKD where everything ships as containers, it makes downloading the massive amount of container images more efficient if teaming was enabled from the get go. Is that correct?

Anyway, not opposed overall. Size-wise, from a rough eyeball calculation, it looks like about 1M total of added software. While FCOS can diverge from other Fedora editions, I think the fact that it ships by default in Fedora Server is a good indication that it's a stable and important path. Can we add that to the commit message?

[cgwalters]

So we agree teaming is not required on the first boot.

I think so yes.

But e.g. in the case of OCP/OKD where everything ships as containers, it makes downloading the massive amount of container images more efficient if teaming was enabled from the get go. Is that correct?

Maybe, depends on whether teaming is used active/active or active/backup. And we're pulling containers from the main root, and presumably we'd have teaming set up before we get to the point of doing container pulls.

Unless we're talking about containerizing teamd (but I don't think anyone is?)

[cgwalters]

I think the fact that it ships by default in Fedora Server is a good indication that it's a stable and important path. Can we add that to the commit message?

Done.

[cgwalters]

That said, we're not adding any tests for this, and I didn't test this personally, just extracted it from an RHCOS PR.

I will admit in my testing over the past few days I've found the dracut support to have some question marks. It seems to bring up teaming in the initramfs fine, but when switching to the real root the teamd process that was started in the initrmafs to support the interfaces was gone.

For RHCOS we tear down all networking today right?

[cgwalters]

Thanks for all the discussion here BTW, definitely a team effort!

[mrguitar]

Looks like I'm late to this so it's probably moot. Teaming is nice from a configuration standpoint, but I'm not sure this is advantageous for an immutable system. There's no performance benefits and it's basically the same options as bonding. So I'm wondering if there's any value here outside of a feature checkbox. Does NIC teaming buy us something useful w/ FCOS?

[dustymabe]

That said, we're not adding any tests for this, and I didn't test this personally, just extracted it from an RHCOS PR.

Yep. I did test it in RHCOS in the scenario where you use a single NIC in the initramfs and have ignition lay down config files to have NM bring up a teaming device (in my case team0).

I will admit in my testing over the past few days I've found the dracut support to have some question marks. It seems to bring up teaming in the initramfs fine, but when switching to the real root the teamd process that was started in the initrmafs to support the interfaces was gone.

For RHCOS we tear down all networking today right?

yes, we try. Though we do have some special casing for bonding, which I think we'd need to also do for teaming IF we wanted to support bringing up teaming in the initramfs. I think in the very short term my recommendation is going to be to not bring up teaming in the initramfs and just lay down files via ignition to bring it up in the real root. The intermediate term we'll hopefully have NM in the initrd and this won't be an issue any longer, but we'll see.

[dustymabe]

Looks like I'm late to this so it's probably moot. Teaming is nice from a configuration standpoint, but I'm not sure this is advantageous for an immutable system. There's no performance benefits and it's basically the same options as bonding. So I'm wondering if there's any value here outside of a feature checkbox. Does NIC teaming buy us something useful w/ FCOS?

Unfortunately I'm not enough of a networking expert to say. We're mostly responding to http://bugzilla.redhat.com/1758162.

[miabbott]

My two cents is that if this gets us closer to networking feature parity with Fedora/RHEL, I think that is A Good Thing ™️ . The trade-off of adding more packages and growing the size of the OS seems reasonable.

So I'm wondering if there's any value here outside of a feature checkbox. Does NIC teaming buy us something useful w/ FCOS?

FWIW, the downstream BZ was filed by a RHT solutions architect, there's no customer cases attached to the BZ, no other customer BZs filed for teaming support...might just be a checkbox in this case.

[cgwalters]

I think part of this is that from the kernel/networking team PoV, if anyone has feature requests (or bugs) for bonding they'd ask "why aren't you teaming"? And generally a trend is moving some more "control plane" logic into userspace, see also eBPF.

[dustymabe]

we discussed this in the meeting today:

12:08:55      jlebon | #agreed we will ship baked-in support for teaming in FCOS to make network configuration easier on first boot

I'll wait an hour to see if there are any final comments before merging.

[jlebon]

Thanks for all the discussion here BTW, definitely a team effort!

Truly a bonding experience!