Researchers release HOPE expert pruning method

Researchers have introduced HOPE (Higher-Order Pruning of Experts), a second-order pruning method for Mixture-of-Experts language models that accounts for pairwise expert interactions instead of evaluating experts independently. By solving a quadratic program over a calibration dataset, HOPE preserves cooperative groups of co-activated experts and achieves superior performance, particularly at aggressive pruning rates up to 50% on agentic coding tasks. Open article →