![[2404.13895] Optimal Design for Human Preference Elicitation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2404.13895] Optimal Design for Human Preference Elicitation
Summary
The paper discusses optimal design strategies for eliciting human preferences, focusing on efficient methods for gathering high-quality feedback to improve AI models.
Why It Matters
As AI systems increasingly rely on human feedback for training, understanding how to efficiently gather this data is crucial. This research offers insights that can enhance the development of preference models, potentially leading to more effective AI applications in various fields.
Key Takeaways
- Introduces a generalized approach to optimal design for preference elicitation.
- Proposes efficient algorithms for both absolute and ranking feedback models.
- Demonstrates practical applications of the algorithms on existing question-answering problems.
Computer Science > Machine Learning arXiv:2404.13895 (cs) [Submitted on 22 Apr 2024 (v1), last revised 16 Feb 2026 (this version, v4)] Title:Optimal Design for Human Preference Elicitation Authors:Subhojyoti Mukherjee, Anusha Lalitha, Kousha Kalantari, Aniket Deshmukh, Ge Liu, Yifei Ma, Branislav Kveton View a PDF of the paper titled Optimal Design for Human Preference Elicitation, by Subhojyoti Mukherjee and 6 other authors View PDF HTML (experimental) Abstract:Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by the cost of obtaining high-quality human annotations, we study efficient human preference elicitation for learning preference models. The key idea in our work is to generalize optimal designs, an approach to computing optimal information-gathering policies, to lists of items that represent potential questions with answers. The policy is a distribution over the lists and we elicit preferences from them proportionally to their probabilities. To show the generality of our ideas, we study both absolute and ranking feedback models on items in the list. We design efficient algorithms for both and analyze them. Finally, we demonstrate that our algorithms are practical by evaluating them on existing question-answering problems. Comments: Subjects: Machine Learning (cs.LG) Cite as: arXiv:2404.13895 [cs.LG] (or arXiv:2404.13895v4 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2404.13895...
