Raspberry Pi 4 onboard NIC Support

[anthr76]

Current situation
Raspberry Pi 4 doesn't have onboard NIC support.

Impact
After chaining off a matchbox configuration flatcar-linux can't reach it's matchbox endpoint after booting into flatcar-linux-installer

See photo

Ideal future situation
Enable the Raspberry Pi 4 networking stack for better Arm64 support.

Implementation options
I'm not a kernel expert but I believe https://github.com/flatcar-linux/coreos-overlay/blob/e4c239ab96ec750717978351f5db6494934d0cec/sys-kernel/coreos-modules/files/commonconfig-5.9#L78 should be Y instead of M.

CONFIG_BROADCOM_PHY=m

The Raspberry Pi 4 interface has a BCM54213PE as noted here on page 11.

According to kernel documentation enabling this option enables support for BCM5421

Additional information

These Raspberry Pis are running UEFI with iPXE to chainload matchbox configured with the Typhoon Kubernetes distribution. After chain loading the latest alpha build if I plug in a usb NIC where the error shows connection refused the installer continues to load but doesn't proceed since the MAC address has changed. Before opening a PR to enable CONFIG_BROADCOM_PHY=Y I wanted to hear some input on this.

[pothos]

The m means it's build as module, so that is not the problem, I think, but there are many RPi related config flags we can check to be m or y and not unset.

[anthr76]

So if it's built it should be being loaded automatically I presume? For reference I'm on the latest alpha build 2671.0.0. For what it's worth it's definitely NIC related. After iPXE exits the onboard NIC no longer blinks for activity.

[pothos]

Yes, the module will be loaded automatically (there is a way to force it via a dracut kernel command line option but I don't expect it to help).
Maybe this here is the list for relevant RPi4 flags? https://github.com/torvalds/linux/blob/master/arch/arm/configs/bcm2835_defconfig Not sure…

[anthr76]

The Raspberry Pi 4 has a Broadcom 2711 perhaps this would be a better basis https://github.com/raspberrypi/linux/blob/rpi-5.8.y/arch/arm64/configs/bcm2711_defconfig this is the Linux 5.8 branch following the Alpha channel.

I can also attempt to dracut the module but looking at this that's likely not the only culprit.

[shift]

I've submitted a patch here flatcar-archive/coreos-overlay#712 which should resolve the NIC issue. I've currently got 10 8GB Pi4s sat waiting to be turned into a cluster, so if any more issues crop up, I'll be submitting changes for them also.

[anthr76]

Awesome to hear @shift ! I will give this a try soon

We have a growing community of running K8s on raspberry pis and at home. I'd love to chat since it seems like it's an incredibly small crowd trying to get flatcar-linux going on RPIs. Is there somewhere we can connect?

[shift]

@anthr76 I'm @shift on Keybase if you use keybase.
I documented some of the steps I took to get TFTP+UEFI booting working on the Pi4 here btw https://gist.github.com/shift/98f43d98ce5e47e374fa46bf67b6a3b1, may be of interest to you.

[matti]

@shift awesome! what's the reason to use TFTP and not just HTTP ipxe...?

[shift]

@matti The net booting doesn't support iPXE booting, so the TFTP server is only to fetch the UEFI and iPXE files. They could be on the sdcard or fetched via TFTP.

[matti]

but what about this? https://github.com/ipxe/pipxe

[wernerb]

Fyi using the latest flatar alpha does not work with the PI in my case, the following when booting with iPXE:

[  132.804736] bcmgenet BCM6E4E:00 enabcm6e4ei0: failed to connect to PHY
[  259.524964] Unable to find mii

Curious if anyone has this working? Im using UEFI->iPXE with 8gb enabled

[dongsupark]

This issue was already fixed via flatcar-archive/coreos-overlay#716.
Will close.
The iPXE boot issue is be discussed in #375 .
Thanks!