[2604.02507] Reinforcement Learning from Human Feedback: A Statistical Perspective

Statistics > Machine Learning arXiv:2604.02507 (stat) [Submitted on 2 Apr 2026] Title:Reinforcement Learning from Human Feedback: A Statistical Perspective Authors:Pangpang Liu, Chengchun Shi, Will Wei Sun View a PDF of the paper titled Reinforcement Learning from Human Feedback: A Statistical Perspective, by Pangpang Liu and 2 other authors View PDF HTML (experimental) Abstract:Reinforcement learning from human feedback (RLHF) has emerged as a central framework for aligning large language models (LLMs) with human preferences. Despite its practical success, RLHF raises fundamental statistical questions because it relies on noisy, subjective, and often heterogeneous feedback to learn reward models and optimize policies. This survey provides a statistical perspective on RLHF, focusing primarily on the LLM alignment setting. We introduce the main components of RLHF, including supervised fine-tuning, reward modeling, and policy optimization, and relate them to familiar statistical ideas such as Bradley-Terry-Luce (BTL) model, latent utility estimation, active learning, experimental design, and uncertainty quantification. We review methods for learning reward functions from pairwise preference data and for optimizing policies through both two-stage RLHF pipelines and emerging one-stage approaches such as direct preference optimization. We further discuss recent extensions including reinforcement learning from AI feedback, inference-time algorithms, and reinforcement learning fr...