A Megakernel per Model

dangxingyu.github.io21 min read

Researchers demonstrate that AI coding agents can write specialized megakernels for individual model checkpoints on GPUs, running entire decode loops without returning to the host between tokens. Tested across eight models on H100 and B200 GPUs, these single-kernel engines achieve significant speedups and maintain higher service-level objective attainment compared to general engines like vLLM and SGLang under production request traces. Open article →