Objective-Conditioned Active Preference Learning for Multi-Objective Alignment
arXiv:2611.05506v1 Announce Type: new Abstract: Aligning large language models (LLMs) with several objectives, such as helpfulness, harmlessness, and honesty, typically relies on preference judgment for each objective, growing the annotation cost linearly with the number of objectives. Active preference learning can reduce this cost by selecting the comparisons whose labels are expected to change the policy most. Existing remedies either lack an objective-aware query selection mechanism for the multi-objective setting or regard conflicting preferences as harmful, therefore excluding them from
阅读原文 · arXiv 机器学习本文由 SoccerDream AI 自动聚合索引，内容版权归原作者与 arXiv 机器学习 所有。
本站仅作技术动态聚合，不构成任何形式的建议。
本站仅作技术动态聚合，不构成任何形式的建议。