Communicate the Change: Efficient Expert Parallelism via Activation Compression
Researchers introduce EchoMoE, a lightweight architecture modification for large-scale mixture-of-experts language models that addresses interconnect communication bottlenecks during inference. By communicating only compressed differences relative to reference states from earlier transformer blocks, EchoMoE reduces inter-node communication volume by 58% and improves throughput by 22.6%. The method preserves average performance within 3.8 percentage points across 18 benchmarks and integrates with vLLM. Open article →