Cursor open sources megakernel for massive mixture of experts scaling

A single, deterministic kernel now handles the entirety of communication and computation for Mixture-of-Experts models on Nvidia’s NVL72 systems. Released today by Cursor, the Mixture-of-Kittens kernel achieves up to 2.37 times higher throughput than existing public baselines. For operators managing large-scale inference and training, the optimization translates to a 41 percent increase in tokens per second—a metric that, at industrial scale, dictates the viability of frontier model deployments.

The launch arrives amid a fundamental debate over the future of inference engineering. Proponents of kernel fusion argue that hand-coding massive, 60,000-line kernels is necessary to eliminate the overhead of launching individual operations and to bypass the bottlenecks of inter-kernel data transfer. However, as GPU architectures evolve, the hardware itself is beginning to automate the coordination that megakernels were designed to solve. Nvidia’s Rubin architecture, for instance, introduced tile-level dependency triggers, allowing a second operation to begin as soon as partial data from the first is available, rather than waiting for a full kernel to clear.

Despite these hardware advancements, the Cursor release demonstrates that manual fusion still offers a performance ceiling that modular libraries like TensorRT-LLM have yet to reach. By fusing all communication into a single pass, Mixture-of-Kittens bypasses the latencies inherent in tensor parallelism, where GPUs must frequently stall to synchronize partial results. While the complexity of maintaining such kernels remains a barrier for most production teams, the efficiency gains suggest that for the industry’s largest players, the overhead of hand-fused engineering is a price worth paying for raw throughput.