MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs
arXiv:2610.30206v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model capacity efficiently by activating a small subset of parameters per token. However, in long-context inference, memory remains a major bottleneck because the KV cache grows linearly with sequence length. Existing KV cache compression methods reduce this cost for dense transformers, but overlook a problem specific to MoE: KV eviction can change which experts are selected during answer generation, degrading the generation quality. We propose MoEZip, a routing-aware KV cache compression for sparse
阅读原文 · arXiv 机器学习本文由 SoccerDream AI 自动聚合索引，内容版权归原作者与 arXiv 机器学习 所有。
本站仅作技术动态聚合，不构成任何形式的建议。
本站仅作技术动态聚合，不构成任何形式的建议。