Do Mixture-of-Experts Routers Benefit from Ensembling?
arXiv:2611.17472v1 Announce Type: new Abstract: Ensembles of neural networks have repeatedly been shown to outperform single networks. Ensemble theory attributes this gain to the accuracy of the individual members and the diversity among them. Mixture-of-experts models already combine many experts, but a single router decides how the experts are weighted for each input. Motivated by ensemble theory, we apply ensembling to the routing step: an ensemble of routers, trained jointly with the experts, proposes the expert weights. We systematically evaluate this approach using underparameterized net
阅读原文 · arXiv 机器学习本文由 SoccerDream AI 自动聚合索引,内容版权归原作者与 arXiv 机器学习 所有。
本站仅作技术动态聚合,不构成任何形式的建议。
本站仅作技术动态聚合,不构成任何形式的建议。