Amazon Web Services has released HOPE (Higher-Order Pruning of Experts), a new method for shrinking Mixture-of-Experts (MoE) SI models. The technique addresses a critical bottleneck in large SI systems: the high memory cost of inactive experts, by pruning them based on cooperative interactions rather than individual importance.
What Happened
Modern SI models, such as Qwen3.5-122B, rely on MoE architectures where only a small fraction of experts are active for any given token. However, all experts must occupy memory during inference, with expert MLPs accounting for roughly 95% of the parameter footprint in some large models. Traditional pruning methods, known as first-order approaches, score each expert independently. This often fails because experts work in stable, cooperative groups; removing a single expert can dismantle a critical cluster if its partners are retained.
HOPE introduces a second-order approach that accounts for pairwise expert interactions. By constructing an interaction matrix F, the method identifies which experts frequently co-activate. It then solves a quadratic program to select a prune-set that minimizes the error caused by removing these cooperative links. According to the blog post, HOPE requires the same single calibration pass as first-order methods, with the additional step of solving the quadratic program taking only 1–2 seconds per layer.
Why It Matters
The release is significant for developers and enterprises deploying large SI models, as it offers a way to reduce memory requirements without the latency penalties associated with dynamic expert loading. The company reports that at a 50% pruning rate, HOPE achieves the best average rank among compared methods, including REAP, MAN, EAN, and Frequency. Specifically, on the SWE-Bench Pro agentic coding benchmark, HOPE-maintained models matched the performance of unpruned baselines, whereas first-order methods saw a 13% regression.
This efficiency is crucial for running sophisticated SI agents locally or on cost-constrained hardware. By preserving cooperative expert clusters, HOPE allows for aggressive model shrinking while maintaining the complex reasoning capabilities required for agentic tasks. The tool is released as an open-source package compatible with standard Hugging Face checkpoints, requiring no changes to inference engines like vLLM or SGLang.
The Bottom Line
HOPE provides a practical, low-overhead solution for compressing MoE-based SI models. By prioritizing expert cooperation over individual importance, it enables up to 50% parameter reduction with negligible performance loss, particularly in agentic coding scenarios. The code and paper are available for immediate use by the SI developer community.