Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

Dang, here I was feeling slick with 10Gb between my MBP and Mac mini and NAS. That’s the thing, it could always be bigger / faster / more. It’s enough for my workflow, but it’s fun to see people push further.

One of the YouTube comments pointed out the issue is most likely the lack of support for SMB Direct (RDMA) in macOS. Would be nice to repeat the test on a Windows/Linux laptop.

It’s crazy that the Ethernet chips consume more power (presumably - because of the heat) than the processor in the computer. How is driving 25Gb/s such a hot task compared to driving multiple 40Gb/s Thunderbolt ports?

What are typical use cases for needing 25Gbps?

Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps.

I'm not saying it's not needed, - just curious.

Somewhat off-topic: has anyone implemented USB C based networking?

So for instance you have two or more machines that have USB 4 ports at 20 to 80 Gbps and you plug in the appropriate USB C cables to connect them.

What do you need to make the connections appear as a (point to point) part of a regular TCP/IP network? Seems much cheaper than Ethernet if you already have the ports.

One of the YouTube comments pointed out the issue is most likely the lack of support for SMB Direct (RDMA) in macOS. Would be nice to repeat the test on a Windows/Linux laptop.

https://developer.apple.com/documentation/technotes/tn3205-l...

MacOS has supported RDMA over thunderbolt since March 19th.

This is a ConnextX4 on a thunderbolt bridge. This is well trodden territory. Unless some hero of open source has come to bat recently, MacOS doesn't support RDMA with this card or over Ethernet at all for that matter. It's only supported over Thunderbolt.

His iperf3 results show that it's able to go full beans on a single 25g port. A cross platform SMB file copy is kind of a meaningless benchmark in this context.

IMO a pretty cool product would be a network switch providing native thunderbolt interfaces to multiple upstream hosts. That wouldn't magically fix this RDMA issue, though. This is just a separate thought. Seems like the kind of thing Mikrotik should make.

I have a 25GbE setup using Thunderbolt, kinda similar to the author's. Except that my fan is held together by a piece of string, not a 3D-printed enclosure.

I found that SMB on macOS is just bad. If I try to heavily load it (e.g. by doing mass conversions in my Audible library), it can cause stalls and disconnections.

I managed to find a set of parameters (in /etc/nsmbd.conf) that makes it work more-or-less reliably by trial-and-error: dir_cache_off=yes, file_ids_off=yes, notify_off=yes, mc_on=no.

It's still terrible, though. It's so bad that Linux in a Parallels VM works _better_.

As I understand it, it takes a bit of anger to get electromagnetism to wiggle that fast over copper. Imperfect transistors using their full operational envelope to control a sort of chaotic thing? Fields and stuff.

They’re not that bad, but even a couple watts of power with no way to dissipate is going to get hot enough to burn you. Forced air cooling over a heatsink is really effective. 15W under load is much lower than a high end desktop CPU that can pull 150-200W.

See also NVMe drives? They draw enough idle power to drain a laptop battery overnight, and will easily hit 60+C in a bad enclosure. That’s only a 10Gbps module. The USB4 enclosures are all heatsink.

A Mac Studio is over 100 W while a 25G NIC is under 25 W (that's the power limit for a normal PCIe slot).

Maybe the NIC does a lot? In past experience, a whole Xeon CPU couldn't keep up with a 10gig NIC if it was sending/receiving min-size UDP packets. Even with Intel DPDK iirc.

Something like 5 percent of AI datacenter usage is just optics.

It's distance. 25GbaseT is good for 30m+. Thunderbolt is good for 1m without active cables. Making the much more rugged signal is a much more intense job.

And your computer already speaks the PCIe and thunderbolt. It has that probably on CPU. Here we have to receive USB, convert it to Ethernet. On much less up to date process nodes than your CPU.

Third, until very recently for even 10Gbit it meant using a usb4<->pcie bridge chip which itself was hot, to provide a generic pcie connection, which then talked to a conventional network chip, so it was a double hop: USB <-> pcie <-> nic. I don't know if 25Gbit to USB direct convert is available yet but we've seen a wave of vastly cheaper 10Gbit adapters come out that are single chip USB<->nic, probably not pcie at all, that are much much cooler. 25Gbit is probably still not here yet, maybe?

Ethernet is isolated through transformers on the physical layer and they're always sinking power. Thunderbolt is just pushing current directly through.

With ethernet it's for lots of reasons but a big one is not creating ground loops between distant equipment. If you didn't isolate the network cable would be sinking potentially amps of current between distant devices and that just isn't sustainable. You don't have this problem attaching USB between two devices.

What are typical use cases for needing 25Gbps?

Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps.

I'm not saying it's not needed, - just curious.

Running local LLMs split across machines. Pretty sure the author has some other posts about doing that.

Edit: https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-stu...

We refurbish laptops and hence distribute .iso / ipsw images to various refurbishing stations. (or also, think of PXE/network boot situations). More images x more laptops refurbished at once = more bandwidth.

Obviously it's possible to move this to a NAS, not use a mac, etc. etc. - but just wanted to chime in with my little use case of moving multi-gig-files around. :-)

I was just looking for a thunderbolt-sfp solution. Super timely!

> Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps.

That'd be the rate for 1 stream at 1x playback. Multiple streams because of overlays/crossfades/etc... along with playback at >1x (such as during rendering in particular) will change that significantly.

What if it’s uncompressed footage at 60fps? And dealing with many shots. And on top of that maybe working with multiple projects in parallel.

If you're editing video you're not just streaming in real time one copy. You have many streams that you're going to want to work around quickly.

Somewhat off-topic: has anyone implemented USB C based networking?

So for instance you have two or more machines that have USB 4 ports at 20 to 80 Gbps and you plug in the appropriate USB C cables to connect them.

What do you need to make the connections appear as a (point to point) part of a regular TCP/IP network? Seems much cheaper than Ethernet if you already have the ports.

I used to have a thunderbolt ring network for a ceph backend on my k8s cluster. On Linux load the thunderbolt_net kernel module and they show up the same as any other Ethernet interfaces. Just directly connect them and configure the network settings on each end using your normal tools.

Latency was an order of magnitude higher than twisted pair but the throughput was greater than 10Gbps and I didn’t have to spend on 10GbE capable servers and network equipment. I vaguely remember that they had to be proper Thunderbolt not just USB-C but that could have changed it’s been quite a while.

Yup, totally doable. Of course, it’s a point to point link so if you want to build a whole cluster that way then you’ll need a lot of usb-c ports or you’ll need to make more of a mesh topology.

I believe MacOS supports it via the usual network control panel. I’ve used it for transferring data from an older laptop to a newer one. The cable is more expensive than fiber, last time I checked.

USB-4 is basically Thunderbolt right? Thunderbolt networking is pretty automatic on Macs at least: https://support.apple.com/en-gb/guide/mac-help/mchld53dd2f5/...

With USB 4, very strictly speaking: It's iffy. USB 4 doesn't require hosts to be able to switch roles and talk to other hosts. It can support this, but support is optional.

But for all certified Thunderbolt 4 and 5 ports, this kind of networking support is a requirement. (It is optional with Thunderbolt 3.)

The rest is up to the operating system.

Yes, there are a few standard(ish) mechanisms for doing this, like [RNDIS][1] and NCM. If you have two Thunderbolt-equipped Macs and connect their ports together, you can use Bonjour/zeroconf for network discovery between them.

[1]: https://en.wikipedia.org/wiki/RNDIS

His iperf3 results show that it's able to go full beans on a single 25g port. A cross platform SMB file copy is kind of a meaningless benchmark in this context.

I have a 25GbE setup using Thunderbolt, kinda similar to the author's. Except that my fan is held together by a piece of string, not a 3D-printed enclosure.

I found that SMB on macOS is just bad. If I try to heavily load it (e.g. by doing mass conversions in my Audible library), it can cause stalls and disconnections.

I managed to find a set of parameters (in /etc/nsmbd.conf) that makes it work more-or-less reliably by trial-and-error: dir_cache_off=yes, file_ids_off=yes, notify_off=yes, mc_on=no.

It's still terrible, though. It's so bad that Linux in a Parallels VM works _better_.

https://developer.apple.com/documentation/technotes/tn3205-l...

MacOS has supported RDMA over thunderbolt since March 19th.

That is just RDMA itself and seems to be aimed directly at distributed inference (no surprise there!). Now they need to update the SMB driver to make use of it.

Is RDMA something worth looking into for connecting two random Linux computers? I've been extremely underwhelmed by the stability and performance of my NFS shares.

is thunderbolt not copper?

can confirm. anger causes loud fast communication that can be heard over background noise. I've heard anger using light-based signalling like sign language might be more efficient.

They get hot using fiber as well.

Maybe the NIC does a lot? In past experience, a whole Xeon CPU couldn't keep up with a 10gig NIC if it was sending/receiving min-size UDP packets. Even with Intel DPDK iirc.

Something like 5 percent of AI datacenter usage is just optics.

It's distance. 25GbaseT is good for 30m+. Thunderbolt is good for 1m without active cables. Making the much more rugged signal is a much more intense job.

And your computer already speaks the PCIe and thunderbolt. It has that probably on CPU. Here we have to receive USB, convert it to Ethernet. On much less up to date process nodes than your CPU.

Ethernet is isolated through transformers on the physical layer and they're always sinking power. Thunderbolt is just pushing current directly through.

Running local LLMs split across machines. Pretty sure the author has some other posts about doing that.

Edit: https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-stu...

Obviously it's possible to move this to a NAS, not use a mac, etc. etc. - but just wanted to chime in with my little use case of moving multi-gig-files around. :-)

I was just looking for a thunderbolt-sfp solution. Super timely!

> Author talks about 4k video, but AI tells me that for "4K production mastering (ProRes 422 HQ, 4:2:2, 10-bit)" the rate is under 2Gbps.

If you're editing video you're not just streaming in real time one copy. You have many streams that you're going to want to work around quickly.

USB-4 is basically Thunderbolt right? Thunderbolt networking is pretty automatic on Macs at least: https://support.apple.com/en-gb/guide/mac-help/mchld53dd2f5/...

With USB 4, very strictly speaking: It's iffy. USB 4 doesn't require hosts to be able to switch roles and talk to other hosts. It can support this, but support is optional.

But for all certified Thunderbolt 4 and 5 ports, this kind of networking support is a requirement. (It is optional with Thunderbolt 3.)

The rest is up to the operating system.

[1]: https://en.wikipedia.org/wiki/RNDIS

can confirm. anger causes loud fast communication that can be heard over background noise. I've heard anger using light-based signalling like sign language might be more efficient.

They get hot using fiber as well.

is thunderbolt not copper?

A Mac Studio is over 100 W while a 25G NIC is under 25 W (that's the power limit for a normal PCIe slot).

A normal x16 PCIe slot has always been able to supply up to 75w since the beginning.

What if it’s uncompressed footage at 60fps? And dealing with many shots. And on top of that maybe working with multiple projects in parallel.

Normal 4K editing workflows are fine with even 2.5 Gbps wired networking (wired to make latency like when scrubbing not-horrid)... 10 Gbps more necessary if editing ProRes RAW and doing multicam (4-8 streams simultaneously).

I'm in a never-ending pursuit of cutting time when transferring large files though — AI models is one thing, but I have 100-200 GB projects I transfer across for backup or import (using an external TB5 drive capable of 3-5 GB/sec), and if I can do that in 1 or 2 minutes per project instead of 3 or 4, that's a noticeable speedup.

Definitely overkill for 98% of my workflow though. 10 Gbps is fine.

Yup, totally doable. Of course, it’s a point to point link so if you want to build a whole cluster that way then you’ll need a lot of usb-c ports or you’ll need to make more of a mesh topology.

I believe MacOS supports it via the usual network control panel. I’ve used it for transferring data from an older laptop to a newer one. The cable is more expensive than fiber, last time I checked.

That is just RDMA itself and seems to be aimed directly at distributed inference (no surprise there!). Now they need to update the SMB driver to make use of it.

Is RDMA something worth looking into for connecting two random Linux computers? I've been extremely underwhelmed by the stability and performance of my NFS shares.

I have also used a USB 4 cable to network two MacBook Pros to one another and can confirm that it was very simple to connect and it was fast as heck. The USB 4 cable I used was rated for up to 40 Gbps.

Yeah, raw RDMA is one thing, making sure other protocols can use it is another... hopefully Apple gets there, but right now SMB with RDMA is probably a tiny, tiny niche.

I do wonder what Apple's custom-built servers are running for networking. I wish they still made Mac Pro-level chassis specifically for better IO. Thunderbolt 5 is nice, but far inferior to a PCIe x16 slot (which can hit 400 Gbps networking) nowadays.

Yes, nfs with rdma will increase your performance significantly.

A normal x16 PCIe slot has always been able to supply up to 75w since the beginning.

Definitely overkill for 98% of my workflow though. 10 Gbps is fine.

Yeah, raw RDMA is one thing, making sure other protocols can use it is another... hopefully Apple gets there, but right now SMB with RDMA is probably a tiny, tiny niche.

Yes, nfs with rdma will increase your performance significantly.

If your storage devices are native PCIe and your network is actually fast enough and your nics and switches have the proper hardware and driver support and your switches are properly configured and your nfs client and server configurations are tweaked just so, and you hold your face in just the right position

The perf is amazing

Mac Studio with all ports plugged in Ethernet 10G

I've been using the built-in 10 Gigabit Ethernet on my Mac Studio for a few years. It works fine: I can edit 4K video straight off my NAS over the network, and run backups at around 1 GB/sec.

But... I want more. I upgraded my rack and my NAS to 25 GbE a couple years ago, and wanted to upgrade my main workstation, too.

I looked up 25G networking options for the Mac, and they're all crazy expensive:

Sonnet's Twin25G Adapter is $999
Atto's ThunderLink is $1,099
Raiden Digit's LightOne 25GbE Docking Station is $399 (which honestly, isn't bad)

The problem is Macs all require Thunderbolt adapters; you can't just plug an inexpensive(ish) PCIe card into a Mac (RIP Mac Pro).

I stopped looking until I saw this blog post. Christian Kohlschütter found a cheap 25G Thunderbolt adapter that uses a server-pulled OCP 2 NIC with a little Thunderbolt 3 adapter board. And it works on any computer with Thunderbolt.

I paid $166.71 in January for the 25G Thunderbolt NIC

Back in January, it was only $160, which was insta-buy territory for me. Since that time, the Amazon listing jumped to $299, which still might be good compared to the Sonnet... but you might have to go digging through some Chinese sites to find a non-marked-up version now.

This blog post is a companion to the following YouTube video:

First Test - Two Problems to Solve

Once I had pulled some new fiber to my desk (where I only had Cat6A cabling before), I tested the bandwidth using iperf3 between my Mac and my NAS. That's when I ran into two problems:

The version of iperf3 I was running on the NAS was too old. Without multi-threading, it maxed out at 15 Gbps.
The 25G NIC enclosure was getting hot. Painful to the touch.

I could solve the first problem easily: I compiled the latest version of iperf3. That got me to 20 gigabits, since more than one CPU core could hand the transfers on my NAS. As Christian mentioned in his blog post, 20 Gpbs single direction and 25 Gbps bidirectional is about the limit for the Thunderbolt 3 chipset being used (even if you plug into a Thunderbolt 5 port).

Thunderbolt 3 to PCIe NIC OCP 2 adapter board

But the second problem was more tricky.

The burning-hot enclosure wasn't thermally bonded to the OCP 2 network card, meaning the NIC chips were cooking.

They had tiny heatsinks on them, but OCP 2 NICs are meant to be inside servers with high pressure fans, not in a little passively-cooled enclosure.

It was acting like a little oven.

Fixing the NIC's cooling problem

Christian mentioned he slapped a couple giant heatsinks on the enclosure. That brought the chip down to a temperature that wouldn't cause NIC dropouts, but it was still getting pretty hot.

I wanted to make sure things were stable, and that meant active cooling.

My first idea was to stick on these low-profile heatsinks and set this speed-controlled USB fan in front. I had to remove the enclosure's barely-ventilated front plate to get more airflow inside, but the back plate also created a ton of resistance.

It would still get hot, and the fan was just loud enough to be distracting, even on its lowest setting.

A Prusa rep had reached out around this time asking if I had any use for their new Prusament PLA in Noctua Brown and Beige... and I decided to switch tracks once they offered to send a spool of each. I purchased a Noctua NF-A8 80mm 5V fan, and an NA-FC1 Speed Controller to silence it.

I designed a fan duct for the 25G Thunderbolt NIC enclosure, and printed it in Noctua beige PLA.

Then I printed this airflow-optimized 80mm fan grill, and screwed that on the front of the 80mm fan.

I was able to use the screws from the 25G NIC enclosure (I removed the front plate entirely), and the fan screws and extra cable that came with the Noctua 80mm fan, to secure everything together.

I chose to splice the braided fan extension cable Noctua includes, and taped it down inside the enclosure with kapton tape for some strain relief.

I soldered the cut end of the fan cable into these through-holes on the Thunderbolt-to-OCP adapter PCB to get the needed 5V power (well, 4.8V, but it's close enough):

Soldering the fan connector on the NIC adapter board for 5V power

The fan only used about 0.5W of power, so I don't think it'll cause any brownout conditions on the NIC itself (which uses 4-5W total at idle).

After final assembly, this is what it looks like:

Holding the finished NIC with a Noctua 80mm fan and grill cover

I plugged it in, re-tested with iperf3, and checked the temperature. It was sitting at less than 36°C after 10 minutes, with the fan on low. And this being a Noctua fan, I couldn't hear it at all under the desk.

How does 25 Gbps perform on the Mac?

Like earlier, it maxes out around 20-25 Gbps, because of the slower Thunderbolt 3 connection.

Activity Monitor showing 1.43 GB/sec on macOS with 25G NIC

Testing Samba file copies between my NAS and my Mac, I got around 1.4 GB/sec read, and 1 GB/sec write1.

That's only marginally better than the built-in 10G Ethernet. It's an improvement, sure... but was all the work pulling fiber, designing a fan cowling, paying $200 for all the parts, and assembling everything worth it?

Maybe. At least I got this blog post out of it.

I have SMB multichannel enabled, but this real-world speed limitation could also be related to my use of my lower-power Arm NAS (running on Ampere Altra, with 32 fairly-slow CPU cores). I was testing writing to an array of fast enterprise NVMe SSDs, so they shouldn't be the bottleneck here. ↩︎

Hacker Times