Cursor Research has released Mixture-of-Kittens, or MoK, as Apache-2.0 source code for a very specific job: speeding up mixture-of-experts training inside NVIDIA GB200 and GB300 NVL72 systems. The August 4 announcement says the kernel now underpins Cursor’s Composer training across tens of thousands of GPUs, while the public repository confirms that it covers forward and backward MoE execution in BF16 and MXFP8.
The practical news is narrower than the “open-source” label suggests. MoK is deployable source code, but it is not broadly deployable infrastructure. Its supported hardware floor is NVIDIA Blackwell SM100 or SM103, with the project explicitly naming GB200 NVL72 and GB300 NVL72 installations; it also requires Python 3.12 or later, PyTorch 2.10 or later built for CUDA 13.0, and CUDA toolkit 13.0 or newer. That places it outside the reach of Hopper clusters, most enterprise GPU estates, consumer RTX machines, and the ordinary eight-GPU training server.
MarkTechPost first highlighted the release, but its report closely follows Cursor’s own technical post and repository documentation. As of August 5, no independent outlet appears to have published a reproduction of Cursor’s benchmark results. The code and benchmark harness are public, which makes verification possible, but the claimed gains remain vendor measurements until another operator runs the same workloads on comparable NVL72 hardware.
MoE models route each token to a small set of specialized feed-forward networks, or experts. That sparse design reduces the compute required per token relative to running every parameter, but it introduces a severe distributed-systems problem: each GPU must send routed tokens to the GPUs holding the chosen experts, wait for those expert computations, and then collect the results.
Cursor says that communication, rather than the expert matrix multiplications themselves, became the limiting factor in Composer’s production training. Its assessment is plausible for large expert-parallel jobs: a training stack can have plenty of tensor-core capacity while GPUs wait on all-to-all token exchanges and completion signals. The catch is that MoK attacks this problem in the environment where it is most acute—72 tightly connected Blackwell GPUs sharing an NVLink domain—not across the heterogeneous clusters most organizations actually operate.
MoK’s central design choice is a megakernel: instead of launching separate CUDA work for routing, token movement, grouped expert GEMMs, activation functions, and result collection, it fuses those stages into a larger persistent kernel. Some streaming multiprocessors are assigned to communication and some to expert computation, allowing one minibatch to be processed while another arrives.
That fusion is why MoK is more than a replacement call for NCCL or DeepEP. Replacing an all-to-all library alone would not reproduce the result, because MoK’s claimed performance depends on a schedule shared among dispatch, combine, and both backward-pass equivalents. A platform team considering it needs to treat the project as a replacement for the MoE layer’s execution path, not as a package to add beside an existing DeepEP or Megatron deployment.
DeepEP and related approaches generally lean on push-style communication for token movement. Cursor reports that push moves fewer aggregate bytes in an ideal transfer but concentrates most traffic in one direction, leaving the opposite direction of a full-duplex NVLink connection underused when routing becomes imbalanced. Its tests found pull-based dispatch delivered up to 29% better NVLink bandwidth utilization under expert imbalance.
The reported latency difference is more revealing. Cursor measured approximately 103 microseconds of signaling cost for push-based dispatch versus 18 microseconds for pull-based dispatch in its multi-node microbenchmarks. The company’s solution uses pull dispatch in the forward pass, push combine after expert computation, then mirrors that pattern during backpropagation.
The stated advantage is that pulling lets the destination rank request the data and consume it immediately, without waiting for completion notifications from up to 71 peers. In a rack-scale MoE job, avoiding repeated cross-GPU signaling can matter as much as raw bandwidth. Cursor also builds a single schedule table, then reuses it across forward dispatch, forward combine, reverse combine, and reverse dispatch; it says schedule construction consumes less than 3% of MoE runtime and stays device-side.
This is a design worth watching even for teams that cannot run MoK. It makes the useful point that all-to-all traffic should be profiled as a directional and synchronization problem, not reduced to a headline bandwidth number. But the technique relies on the peer-memory behavior and topology of the Blackwell NVL72 environment. It should not be assumed to transfer unchanged to Ethernet-connected GPU nodes, InfiniBand clusters without NVLink fabrics, or PCIe-only servers.
Cursor’s answer is a fixed ring token buffer, described as consuming a few hundred megabytes, that cycles through token minibatches. The kernel interleaves token dispatch from one macrobatch with result combination from another, reusing buffer regions as soon as they are drained. It also walks the ring in reverse macrobatch order to reduce activation recomputation required during backward passes.
This addresses a real GB300 constraint identified by Cursor: the Grace CPU can become visible in the critical path when fast Blackwell GPUs catch up with host-side logging, allocation, metric collection, or synchronization work. Removing the host from buffer sizing lets the GPU work queue run without stopping for those decisions.
Yet “no CPU-GPU synchronization” needs careful reading. It means MoK eliminates that synchronization from its own MoE buffering path. It does not remove the CPU, framework runtime, networking stack, checkpoint process, data loader, FSDP all-gather, or control plane from training. Cursor explicitly describes using Blackwell Cluster Launch Control so its megakernel can yield compute resources for inter-rack RDMA operations such as FSDP communication. The kernel is therefore engineered to coexist with remaining distributed-training overhead, not to make it disappear.
The published baselines were NCCL plus PyTorch, DeepEP plus PyTorch, DeepEP plus TransformerEngine, and HybridEP plus Megatron. Cursor tested shapes modeled after Kimi K2.7 Code, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro. That is a stronger comparison than a synthetic kernel-only chart, because it includes scheduling, dispatch, expert computation, combine, and weighted reduction. It still remains a controlled MoE-layer benchmark designed and run by the project’s authors.
The more operational number is the internal end-to-end result: Cursor says its 512-GPU training run across several GB300 NVL72 racks increased from 760.9 to 1,070.2 tokens per second per GPU after replacing its DeepEP-based MoE path with MoK, a 1.41× increase. That is a meaningful improvement, but it is not a guarantee of a 41% gain for every DeepEP deployment.
Cursor did not publish a full production configuration for that end-to-end test, including model checkpoint, global batch size, sequence length, optimizer settings, exact rack topology, network configuration, data pipeline behavior, or the fraction of total iteration time attributable to MoE on the tested run. Those omissions prevent a third party from determining how much of the gain is portable versus tied to Composer’s model shapes and training stack.
The repository does expose a higher-level functional interface that builds a schedule, creates or reuses a workspace, then invokes forward and backward passes. It also offers a lower-level operations layer for teams prepared to manage symmetric tensors and CUDA kernel calls themselves. The project’s own documentation recommends the functional layer for production training.
That is a usable starting point, but it is not an out-of-the-box integration for a generic PyTorch model. Teams will still need to map their router outputs, routed-expert weights, expert-parallel process group, mixed-precision flow, checkpointing, and distributed training framework onto MoK’s APIs. MXFP8 operation also requires pre-quantized routed-expert weights, while Cursor retains the shared expert in BF16 for training stability.
For Windows-focused administrators, this is firmly a data-center Linux CUDA workload rather than a workstation feature. The first useful validation is not a local install on a Windows PC; it is a controlled test on an eligible NVL72 partition with a DeepSeek-V3-style MoE layer, an established baseline, and enough observability to separate MoE speed from the rest of the training step.
Cursor has made a substantial piece of its production optimization stack inspectable and modifiable. The immediate beneficiaries will be GPU clouds, frontier-model labs, and research centers already paying for Blackwell NVL72 capacity. Everyone else should view MoK as a detailed reference implementation of rack-scale MoE scheduling—and wait for independent runs before treating its 2.37× peak and 1.41× end-to-end figures as portable performance expectations.
The report also identifies several exposed tuning controls: separate forward- and backward-pass communication-SM allocations, minibatch and macrobatch sizes, and schedule-capacity settings. Cursor says these values are workload-dependent and should be tuned before production deployment.
MarkTechPost first highlighted the release, but its report closely follows Cursor’s own technical post and repository documentation. As of August 5, no independent outlet appears to have published a reproduction of Cursor’s benchmark results. The code and benchmark harness are public, which makes verification possible, but the claimed gains remain vendor measurements until another operator runs the same workloads on comparable NVL72 hardware.
The real target is the MoE communication bottleneck
MoE models route each token to a small set of specialized feed-forward networks, or experts. That sparse design reduces the compute required per token relative to running every parameter, but it introduces a severe distributed-systems problem: each GPU must send routed tokens to the GPUs holding the chosen experts, wait for those expert computations, and then collect the results.Cursor says that communication, rather than the expert matrix multiplications themselves, became the limiting factor in Composer’s production training. Its assessment is plausible for large expert-parallel jobs: a training stack can have plenty of tensor-core capacity while GPUs wait on all-to-all token exchanges and completion signals. The catch is that MoK attacks this problem in the environment where it is most acute—72 tightly connected Blackwell GPUs sharing an NVLink domain—not across the heterogeneous clusters most organizations actually operate.
MoK’s central design choice is a megakernel: instead of launching separate CUDA work for routing, token movement, grouped expert GEMMs, activation functions, and result collection, it fuses those stages into a larger persistent kernel. Some streaming multiprocessors are assigned to communication and some to expert computation, allowing one minibatch to be processed while another arrives.
That fusion is why MoK is more than a replacement call for NCCL or DeepEP. Replacing an all-to-all library alone would not reproduce the result, because MoK’s claimed performance depends on a schedule shared among dispatch, combine, and both backward-pass equivalents. A platform team considering it needs to treat the project as a replacement for the MoE layer’s execution path, not as a package to add beside an existing DeepEP or Megatron deployment.
Pull dispatch is the consequential design change
Cursor’s most interesting finding is not merely that it wrote a faster kernel, but that it rejects a one-size-fits-all communication direction. In a push transfer, the GPU that owns a token writes it to the remote GPU. In a pull transfer, the destination GPU loads the token from its source.DeepEP and related approaches generally lean on push-style communication for token movement. Cursor reports that push moves fewer aggregate bytes in an ideal transfer but concentrates most traffic in one direction, leaving the opposite direction of a full-duplex NVLink connection underused when routing becomes imbalanced. Its tests found pull-based dispatch delivered up to 29% better NVLink bandwidth utilization under expert imbalance.
The reported latency difference is more revealing. Cursor measured approximately 103 microseconds of signaling cost for push-based dispatch versus 18 microseconds for pull-based dispatch in its multi-node microbenchmarks. The company’s solution uses pull dispatch in the forward pass, push combine after expert computation, then mirrors that pattern during backpropagation.
The stated advantage is that pulling lets the destination rank request the data and consume it immediately, without waiting for completion notifications from up to 71 peers. In a rack-scale MoE job, avoiding repeated cross-GPU signaling can matter as much as raw bandwidth. Cursor also builds a single schedule table, then reuses it across forward dispatch, forward combine, reverse combine, and reverse dispatch; it says schedule construction consumes less than 3% of MoE runtime and stays device-side.
This is a design worth watching even for teams that cannot run MoK. It makes the useful point that all-to-all traffic should be profiled as a directional and synchronization problem, not reduced to a headline bandwidth number. But the technique relies on the peer-memory behavior and topology of the Blackwell NVL72 environment. It should not be assumed to transfer unchanged to Ethernet-connected GPU nodes, InfiniBand clusters without NVLink fabrics, or PCIe-only servers.
A ring buffer removes a synchronization point, not a memory requirement
MoE routing produces variable numbers of tokens per expert and per GPU. Conventional implementations commonly either reserve enough buffer space for a worst case, drop overflow tokens, or involve the CPU to determine exact buffer requirements. Each option has a cost: wasted memory, altered training behavior, or CPU-GPU synchronization.Cursor’s answer is a fixed ring token buffer, described as consuming a few hundred megabytes, that cycles through token minibatches. The kernel interleaves token dispatch from one macrobatch with result combination from another, reusing buffer regions as soon as they are drained. It also walks the ring in reverse macrobatch order to reduce activation recomputation required during backward passes.
This addresses a real GB300 constraint identified by Cursor: the Grace CPU can become visible in the critical path when fast Blackwell GPUs catch up with host-side logging, allocation, metric collection, or synchronization work. Removing the host from buffer sizing lets the GPU work queue run without stopping for those decisions.
Yet “no CPU-GPU synchronization” needs careful reading. It means MoK eliminates that synchronization from its own MoE buffering path. It does not remove the CPU, framework runtime, networking stack, checkpoint process, data loader, FSDP all-gather, or control plane from training. Cursor explicitly describes using Blackwell Cluster Launch Control so its megakernel can yield compute resources for inter-rack RDMA operations such as FSDP communication. The kernel is therefore engineered to coexist with remaining distributed-training overhead, not to make it disappear.
The headline speedup has two different evidentiary levels
Cursor reports four single-layer peak gains against the fastest public baseline: up to 2.37× for MXFP8 forward execution, 1.78× for MXFP8 backward, 1.92× for BF16 forward, and 1.58× for BF16 backward. Those tests used one GB300 NVL72 rack, expert parallelism of 64, and 2,048 tokens per GPU before routing.The published baselines were NCCL plus PyTorch, DeepEP plus PyTorch, DeepEP plus TransformerEngine, and HybridEP plus Megatron. Cursor tested shapes modeled after Kimi K2.7 Code, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro. That is a stronger comparison than a synthetic kernel-only chart, because it includes scheduling, dispatch, expert computation, combine, and weighted reduction. It still remains a controlled MoE-layer benchmark designed and run by the project’s authors.
The more operational number is the internal end-to-end result: Cursor says its 512-GPU training run across several GB300 NVL72 racks increased from 760.9 to 1,070.2 tokens per second per GPU after replacing its DeepEP-based MoE path with MoK, a 1.41× increase. That is a meaningful improvement, but it is not a guarantee of a 41% gain for every DeepEP deployment.
Cursor did not publish a full production configuration for that end-to-end test, including model checkpoint, global batch size, sequence length, optimizer settings, exact rack topology, network configuration, data pipeline behavior, or the fraction of total iteration time attributable to MoE on the tested run. Those omissions prevent a third party from determining how much of the gain is portable versus tied to Composer’s model shapes and training stack.
Determinism is valuable, but integration is still work
MoK fixes the floating-point operation order so identical input produces bitwise-identical output regardless of hardware scheduling and instruction issue order, according to Cursor. That is unusually valuable for internal ablations and on-policy reinforcement-learning post-training, where a run that cannot be repeated precisely makes it difficult to attribute changes in reward, behavior, or model quality.The repository does expose a higher-level functional interface that builds a schedule, creates or reuses a workspace, then invokes forward and backward passes. It also offers a lower-level operations layer for teams prepared to manage symmetric tensors and CUDA kernel calls themselves. The project’s own documentation recommends the functional layer for production training.
That is a usable starting point, but it is not an out-of-the-box integration for a generic PyTorch model. Teams will still need to map their router outputs, routed-expert weights, expert-parallel process group, mixed-precision flow, checkpointing, and distributed training framework onto MoK’s APIs. MXFP8 operation also requires pre-quantized routed-expert weights, while Cursor retains the shared expert in BF16 for training stability.
For Windows-focused administrators, this is firmly a data-center Linux CUDA workload rather than a workstation feature. The first useful validation is not a local install on a Windows PC; it is a controlled test on an eligible NVL72 partition with a DeepSeek-V3-style MoE layer, an established baseline, and enough observability to separate MoE speed from the rest of the training step.
Cursor has made a substantial piece of its production optimization stack inspectable and modifiable. The immediate beneficiaries will be GPU clouds, frontier-model labs, and research centers already paying for Blackwell NVL72 capacity. Everyone else should view MoK as a detailed reference implementation of rack-scale MoE scheduling—and wait for independent runs before treating its 2.37× peak and 1.41× end-to-end figures as portable performance expectations.
Update: Additional details (August 5, 2026)
Open Source For You notes that MoK builds for SM103 by default, with an environment-variable option for SM100. The project is source-installed rather than distributed as a packaged installer, and it does not document a Windows-specific setup route.The report also identifies several exposed tuning controls: separate forward- and backward-pass communication-SM allocations, minibatch and macrobatch sizes, and schedule-capacity settings. Cursor says these values are workload-dependent and should be tuned before production deployment.
References
- Primary source: MarkTechPost
Published: 2026-08-04T18:38:41+00:00
- Related coverage: bestfriends.org
Saving kittens at Best Friends’ newest nursery
A new kitten nursery opens at the Best Friends Visitor Center in Kanab. Visitors can watch the lifesaving work happen through a viewing window.bestfriends.org - Related coverage: lmsys.org
Deploying DeepSeek on GB300 NVL72: Big Wins in Long-Context Inference - LMSYS Org
As the latest addition to the Blackwell family, the GB300 NVL72 is the most powerful platform for long-context LLM inference. In this blog post, we share our latest progress on optimizing DeepSeek R1-...www.lmsys.org - Primary source: opensourceforu.com
Training Kernel Speeds AI Development - Open Source For You
An open-source training kernel combines communication and computation into a single execution pipeline, helping accelerate
www.opensourceforu.com