Test ASUS XG-C100C 10G PCIe Network Adapter #15
Comments
geerlingguy
changed the title
|
So it is an Aquantia AQN107 based card. Sad that I have not yet received my preordered CM4 and IO board in my country. Otherwise I can test it with a good old Intel X550 10G NIC. I guess the bottleneck is with the PCI-E bus and SoC MSI support regardless. |
|
@UnKnoWn-Consortium - Ooh, I'd be interested to see how you get along with the X550; I was glancing through listings for a few Intel cards too, though that one was a little out of my 'test bench' price range, and I didn't need the extra port. I did end up receiving a few Intel 10G SFP+ cards though, so I'll see how far I can get with them too! But yeah, in the end, I'm pretty sure I'll max things out around 3.2 Gbps on the interface, since that's around the bus limit for the 1x PCIe 2.0 lane on the Pi. We'll see, though! |
|
I don't think CAT5e in a short distance will be a issue. I personally run 10G using CAT5e cable(20m) and get almost full speed(Intel X520). Someone actually tested it in 50m length and found it still work. I would rather switch to SFP+ instead of CAT6+ expensive cable. |
|
@misaka10086 - That's good to know. Half my house is wired with Cat5e, and the other half Cat6 (non-a), which I wired up when I moved in a decade ago using remnants from other projects. I kinda wonder if I could justify going fiber pair for future-proofing... though the basement is still unfinished, meaning ripping out cables and pulling new ones is not too strenuous, just a bit messy. Plus, my security system will never (I hope!) need 10, 20, or 40G networking! |
|
@geerlingguy Optical fiber cables also come with CATs like copper cables. For example, the short-distance multi-mode fiber cables have 5 categories (OM1-5) and they are not exactly backward compatible with each other (OM1<>OM2 and OM3<>OM4<>OM5 only) as the light source evolves and the core size changes. So I guess they may not be as futureproof as one would have hoped for. But, of course, with a top spec speed of 100Gbps, they are unquestionably futureproof speed-wise. |
|
FYI one of the most annoying things about the MikroTik interface for their RouterOS is that to switch to SwitchOS it requires going to the RouterOS web UI (you can also do it via the console), then going to System > Routerboard, then clicking Settings, then you can finally select Switch OS instead of Router OS, and reboot. It took me way too long to figure that out. RouterOS has a lot of flexibility, but toooons of complexity too. A lot for a little 4 port switch (that I just want to use as a switch ;). |
|
All right, finally giving this card its first light. First interesting finding:
|
|
|
Sadly, it seems like Aquantia no longer exists: https://www.aquantia.com/driver-download/ It was purchased by Marvell in 2019. So off to Marvell's site to find drivers! |
|
Looks like https://www.marvell.com/support/downloads.html has Linux drivers for the AQC107, specifically Just noting the directions in the downloaded README seem to be written for an older version of the drivers, before Marvell took over. Ah, acquisitions... |
|
Well that's annoying: So. Like with other cards: kernel recompile time? I think so! |
|
Menuconfig: Recompiling now... |
|
Not necessary. |
|
Hmm... running |
|
Oops. Missed something. I also enabled: I've recompiled and am copying things over to the Pi now. |
|
I got the blinkenlights!!! And: A picture of the setup:
|
|
Good news so far—it seems this driver/hardware does better with whatever optimizations it has than the Rosewill did: That's with MTU 1500, no other modifications. |
And I think we've hit our 1x bus limit. Also going to try overclocking, but I'm less confident that will make a difference since this card/chipset/driver is doing a lot more offload than the other ones I've tested... |
|
Testing bidirectional at MTU 9000 (requires source build of And the run, resulting in 4.18 Gbps of traffic both ways (MUCH better than the 2.5 Gbps card!): |
|
I was going to use I worked around that on the i340-T4 by connecting each interface to a unique Pi and setting up custom networks between each of them. It looks like if both interfaces are on the same network, See esnet/iperf#865 for details. I wonder if it's something weird in the Pi's networking stack. I can't get |
|
Unplugging and replugging cables doesn't seem to show any strange things in the drivers themselves: |
|
Exact same command: FIRST, with both interfaces plugged in: SECOND, with the onboard LAN unplugged: |
|
Linux does weird things in choosing which physical interface to use if they're both connected to the same network. We had some head scratching when trying to debug a wifi issue with QoS. Despite SSH'ing to the IP address associated with the Wifi interface, Linux was choosing to route it over the wired one because it had more bandwidth available or was listed first, or something. |
|
Hmm... so I guess I'll have to test with two separate networks. Luckily I have a pile of 1 Gbps USB 3.0 dongles I can pop onto my Mac :) |
|
Just a couple of comments about the ASUS XG-C100C - I have a couple:
|
|
@NoisyBloke - Thanks for the updates! I have mostly Cat6 cabling around here, but did buy a few Cat8 patch cables (between 6 and 20') that I'll be using for testing. Later on I may go with fiber for my longer runs, but for now my network switch area is only about 15' from my main desk in my office :) Good to know the MikroTik can get 'over 9000' but not quite 16000! |
|
Just referencing the fact that I'm running into similar funny networking benchmark results over in #38 — so I'll have to be a lot more careful with |
|
You should really try the |
|
Almost finished getting my new 10G/2.5G hybrid home network set up. Will get back to testing this card a bit more. |
|
If minimalism and optimization is of interest, it seems that going for SFP+ and fiber makes sense, at least from what I read here. Latency and power consumption are improved, that is what convinced me as I made the choice earlier today for my home lab. I will use DAC cables though, not as good but close. As far as testing Intel cards, I see reasonably priced SFP+ new ones : clones like 10Gtek 's X520-10G-1S , can be found on Amazon around $115 these days. There are some new one like EB-LINK's EB-X520-DA1 for fiber at $100 only. |
|
@NickMihailov - Trying your suggestion, I edited New test result: |
|
Changed it back to not specifying the |
|
Changed the MTU to 9000: And now getting: So it seems we reach a point where maybe packet loss starts hitting — IRQ interrupts also went beyond 50% for a moment, so maybe something is queueing up on the Pi's side at that point. But interesting to see we can reach a short stable period around 3.6 Gbps! |
|
All right, to sum things up, here's a video on the card: 10 Gigabit Ethernet on a Raspberry Pi?!. And here's a blog post with more about increasing the speed of the 10 Gbps connection: Getting faster 10 Gbps Ethernet on the Raspberry Pi. |
Hi, need a little help :) Trying to run this card on CM4 and still no luck. I believe I'm using current kernel using Any suggestions please? |
|
@tu1982 - If you're using the raspberrypi/linux repository at the latest commit, it should have aquantia's driver available, are you sure you're on the latest commit on their rpi branch? I haven't ever gone to 'General setup' nor looked at any options for 'Compile also drivers which will not load'—my testing is usually done with a completely clean checkout of the rpi linux kernel source, and then I run |
|
Hi Jeff,
Yes, I believe so. I'm quite new to RPi but I tried to check modules and found out, there is no aquantia folder/file in /lib/modules/'kernel_version'/kernel/drivers/net/ethernet and /phy. Should be there? I found them only in /lib/modules/'kernel_version'/build/...
01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02) I can't see anything regarding aquantia in
There are probably some errors in journalctl: Can you please see something suspicious? Thank you very much! Tomas |
|
@tu1982 - Ah... it looks like you're running 32-bit Pi OS. I have only tested in 64-bit Pi OS, so maybe that's the issue? I don't know how well the driver will work on a 32-bit system. |
Yes, you are right. Shame on me... It seems it does not work on 32-bit system. Thanks a lot. |
May I ask you how you fixed that? |
Just arrived: the ASUS XG-C100C 10G Network Adapter.
For reference, I was able to coax (hehe, 'coax'... but this is RJ45!) 3.2 Gbps out of the Intel i340-T4 when testing all four 1 Gbps ports at the same time with MTU 9000 jumbo frames. And I could get 2.5 Gbps out of the Rosewill Realtek 8125b.
I'm guessing I'll be able to get 3.2 Gbps max out of this card. On the Mac side of things, I'll be using an OWC Thunderbolt 3 10G Ethernet Adapter, which I plan on using more permanently once I get my in-home 10G network set up—before that, though, I need to figure out if I'm going to stick with 'Cat6 and pray' for short runs, or buy Cat6a, Cat7, or heck maybe even go for Cat8 to replace my existing Cat5e and Cat6 runs.
I have a couple Cable Matters RJ45 Cat7 patch cables to use for testing, and I also have a MikroTik 5-port SFP+ desktop switch, into which I'll insert two FLYPROFiber 10G SFP+ to RJ45 adapters so I can stick with RJ45 for now.
I might try some SFP+ Twinax cables for some later testing, but obviously not with this card, as it's RJ45-only.
The text was updated successfully, but these errors were encountered: