Seq@K: Evaluating Language Models Under Sequential Feedback
arXiv:2610.37859v1 Announce Type: new Abstract: Language-model agents differ fundamentally from prior intelligent systems in their ability to learn from natural language in context. This capability enables agents to iteratively refine their answers in response to feedback, a prevalent mode of operation in modern agentic workflows. However, the community primarily ranks models by metrics such as Pass@1 or Pass@K, which cannot capture how they utilize feedback on prior attempts. We introduce Seq@K, a natural evaluation protocol that measures whether a model can solve a problem within K sequentia
阅读原文 · arXiv 人工智能本文由 SoccerDream AI 自动聚合索引，内容版权归原作者与 arXiv 人工智能 所有。
本站仅作技术动态聚合，不构成任何形式的建议。
本站仅作技术动态聚合，不构成任何形式的建议。