Training Language Models for Introspective Oversight
arXiv:2611.23365v1 Announce Type: new Abstract: Language models increasingly exhibit behaviors that are difficult to detect in transcripts, such as reward hacking, deception, and sycophancy. Asking the assistant follow-up questions about its actions is unreliable, since models are generally not trained to report their own misbehavior. As a result, approaches to oversight typically involve querying external models. These face limitations: external transcript monitoring operates only on visible model outputs, and activation decoders lack full context and are prone to confabulation. We introduce
阅读原文 · arXiv 人工智能本文由 SoccerDream AI 自动聚合索引,内容版权归原作者与 arXiv 人工智能 所有。
本站仅作技术动态聚合,不构成任何形式的建议。
本站仅作技术动态聚合,不构成任何形式的建议。