The Hidden Cost of MoE Diffusion Decoding: Profiling and Fixing the Sampler
arXiv:2611.21852v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) decode by iteratively denoising a full canvas of tokens, so every forward pass runs a per-token sampling and stopping pipeline over the entire canvas-by-vocabulary logits tensor. We profile single-user decoding of a 26B-parameter diffusion Mixture-of-Experts model on a bandwidth-bound edge accelerator (NVIDIA GB10) and report a result that is easy to miss: after weight quantisation, this logits pipeline, not the expert GEMMs, is the single largest consumer of decode time. Unlike autoregressive decoding, wh
阅读原文 · arXiv 机器学习本文由 SoccerDream AI 自动聚合索引,内容版权归原作者与 arXiv 机器学习 所有。
本站仅作技术动态聚合,不构成任何形式的建议。
本站仅作技术动态聚合,不构成任何形式的建议。