Microsoft says the preview lets a compatible inference framework run one model across the combined memory and GPU compute of USB4-connected Windows PCs, using NetworkDirect, the same API Windows uses with datacenter RDMA adapters. The preview is coming soon to Windows Insiders in the Experimental channel (Future Platforms).
What RDMA over USB4 actually does
RDMA (Remote Direct Memory Access) moves data directly between the memory of two systems. It skips most of the traditional networking stack, so latency and CPU overhead are low. Windows has supported RDMA for years through NetworkDirect. Until now that meant dedicated Ethernet or InfiniBand adapters, which are built mainly for servers and rarely show up in consumer or developer PCs.
Microsoft's pitch is that many AI PCs already have USB4 ports. Windows 11 already supports host-to-host networking over USB4 through its in-box connection manager. The new work adds a NetworkDirect provider on top of that, with updated in-box USB4 drivers. Microsoft says this means no RDMA adapter, no switch to configure and no third-party driver to install.
Under the hood, per Microsoft's description:
- The Windows USB4 connection manager builds a peer-to-peer tunnel between each pair of PCs.
- A new provider implements RDMA in software on the USB4 host router's DMA rings. Data moves between registered memory regions without extra software copies.
- Each RDMA link uses two pairs of DMA rings, one for data and one for control. The number of rings and USB4 ports on a PC caps the largest cluster you can build.
- The links run on USB4 ports managed by Windows with Kernel DMA Protection.
The provider, samples and link-test tools are open source. So is a testbed built on a fork of llama.cpp. That fork's RPC backend runs over NetworkDirect instead of TCP, and it has tensor-parallel changes that split a model across PCs. Microsoft says it chose llama.cpp to demonstrate the preview and expects other frameworks to adopt the same APIs. The demonstrated path is the llama.cpp fork. Other frameworks are an expectation, not something shown.
The numbers, and how to read them
Latency. In Microsoft's measurements between two PCs, RDMA over USB4 cut round-trip latency for small messages to about 28–50 microseconds. That is roughly four times lower than Ethernet (UDP) over the same USB4 link. Latency stays similarly low at 40 Gbps and 80 Gbps, and 80 Gbps gives twice the bandwidth.
This is a communication benchmark, not an inference benchmark. It does not mean AI runs four times faster. It does fit the physics of the workload, though. With tensor parallelism the PCs exchange data many times for every generated token, so latency matters more than raw bandwidth. Microsoft also says results depend on the model, the framework build and each platform's memory and GPU.
Partner results. All of these are vendor or Microsoft lab figures, not independent tests.
| Platform | Setup | Model | Result |
|---|---|---|---|
| AMD Ryzen AI Max (Microsoft lab) | 3 mini-PCs, Radeon 8060S, 128 GB unified memory each, 40 Gbps full mesh | Qwen3-235B-A22B, 4-bit | About 18.5 tokens/s |
| AMD Ryzen AI Max+ 395 (AMD's measurement) | 3 PCs | Qwen3-235B-A22B, Q4_K_M, 1,600 context, 2,048 generated tokens | 21.98 tokens/s decode, 117.98 tokens/s prompt processing |
| Intel Core Ultra Series 3 | 3 or 4 PCs, 64 GB at 9,600 MT/s each, Thunderbolt 4, full mesh | GPT-OSS-120B | At 1K context: 49.4 tokens/s (3 PCs), 54.2 tokens/s (4 PCs) |
The 18.5 and 21.98 figures come from different measurements. Don't treat them as a like-for-like comparison.
Intel's results at 1K context show 49.4 tokens/s with three PCs and 54.2 with four. Time to first token was 1.1 s and 1.2 s. At 2K context, decode was 49.4 tokens/s on three PCs and 51.7 on four. Time to first token was 2.9 s and 3.2 s. So the fourth PC adds only a modest gain. At 2K context it adds about 5%, and first-token time gets slightly worse. More nodes help, but with diminishing returns.
The Intel point is the useful one. Microsoft says GPT-OSS-120B doesn't fit in a single 64 GB PC. Clustering is what makes it run on those machines at all.
What you need
- Two or more USB4-capable Windows PCs with USB4 ports managed by Windows. 40 Gbps ports work. 80 Gbps ports such as Thunderbolt 5 offer twice the nominal rate, which can help bandwidth-heavy work like long-context prompts.
- N−1 USB4 ports on each PC for an N-PC cluster. A three-PC cluster needs two ports on each machine.
- One direct USB4 cable per pair of PCs, in a full mesh. Microsoft says daisy-chaining makes performance drop sharply.
- Enough unified memory, memory bandwidth and GPU compute across the cluster for your model.
- The same RDMA-enabled Insider build, preview drivers and NetworkDirect provider version on every PC.
The port math limits scale. A four-PC mesh needs three ports per machine, so laptops with two USB4 ports top out at three nodes.
Microsoft says it has tested select AMD Ryzen AI Max PCs in clusters of up to three, and Intel Core Ultra Series 3 PCs with Thunderbolt 4 in clusters of up to four. It adds that these are "the platforms we've tested so far, not the only ones that work." It has not published a compatibility list.
Getting started, and the security trade-off
The Windows build with RDMA support has not shipped yet. Microsoft says it will update the setup instructions once it does. The enablement steps it has posted are:
- Join the Windows Insider Program and install the preview build from the Experimental Channel (Future Platforms) on each PC, once it's available.
- Open Command Prompt as administrator and run:
reg add HKLM\SYSTEM\CurrentControlSet\Services\Usb4HostRouter\Parameters /v EnableInterDomainRdma /t REG_DWORD /d 1 /f
bcdedit /set testsigning on
- Restart each PC. Test signing requires Secure Boot to be turned off in firmware settings.
- Connect the PCs with USB4 cables in a full mesh, one cable per pair.
- Give each USB4 network interface a static IPv4 address, with each connection on its own subnet.
- Check each link with
ndping. - Build the llama.cpp fork with the Vulkan backend and RPC enabled, then load a model of your choice.
When you're done, run bcdedit /set testsigning off and turn Secure Boot back on.
Test signing and disabled Secure Boot weaken the boot-time protections on a machine. Microsoft explicitly says to use dedicated test PCs, not your everyday device. That advice is sound. Don't do this on a work laptop, and don't do it on a machine that holds credentials you care about. Enterprise admins will also want to note that the feature is off by default and needs a deliberate registry change.
Known limitations
- The transport doesn't retry. If a NetworkDirect connection drops, the application must re-establish it.
- Daisy-chained topologies perform poorly.
- You need to allow traffic on the USB4 network interfaces in the firewall.
- RDMA applications must align memory buffers properly for transfers.
Analysis: who this is for
This is a clever use of hardware many people already own. It also gives AI PCs with large unified memory, such as Ryzen AI Max machines, a way to scale out locally without server networking gear. Microsoft frames it as part of building Windows for hybrid intelligence, where AI work runs on your PC or in the cloud, whichever fits best.
Several caveats apply:
- It's a preview. There is no stated release date. Microsoft says it is exploring bringing the feature to more PCs and silicon partners, and to a future Windows release, based on what it learns.
- The results come from partners and use specific models and quantization.
- Setup is developer-grade. It needs static IPs,
ndping, a custom build and Vulkan-backed llama.cpp. - The security cost is real, even if it's temporary.
- Hardware is the practical limit. You need enough ports and memory, and a handful of mini-PCs isn't cheap.
One community commenter on Microsoft's blog asked whether mixed hardware, such as different NPUs and GPU models, would work. The post as captured doesn't answer that. Microsoft does say any model can be used, but the tested platforms were uniform AMD or Intel clusters.
What's next
Microsoft says it is working with AMD, Intel and the open-source community. It is also exploring how Windows ML and the DirectX Compute Graph Compiler could support native multi-PC AI execution. Those are explorations, not shipping features. Feedback goes through Feedback Hub (Win + F) under "Buses - Lower Power Buses, USB controllers or hubs, UART, HID", or as issues on the provider's repository.
For now, the takeaway is modest but real. Windows is getting a low-latency, open-source way for PCs to share memory over cables you already own. If you're a developer working on distributed local inference, the provider and samples are available now. Everyone else can wait for the Insider build and for a path that doesn't involve turning off Secure Boot.
References
- Microsoft shows massive gains with new Windows USB4 feature that combines multiple PCs Neowin · 2026-11-20T07:50:01+00:00
- Introducing RDMA over USB4® on Windows: run larger local AI models across PCs techcommunity.microsoft.com