[2604.04855] The Role of Generator Access in Autoregressive Post-Training

Computer Science > Machine Learning arXiv:2604.04855 (cs) [Submitted on 6 Apr 2026] Title:The Role of Generator Access in Autoregressive Post-Training Authors:Amit Kiran Rege View a PDF of the paper titled The Role of Generator Access in Autoregressive Post-Training, by Amit Kiran Rege View PDF HTML (experimental) Abstract:We study how generator access constrains autoregressive post-training. The central question is whether the learner is confined to fresh root-start rollouts or can return to previously built prefixes and query the next-token rule there. In the root-start regime, output sampling, generated-token log probabilities, top-$k$ reports, and full next-token distributions along sampled trajectories all reduce to one canonical experiment, limited by the on-policy probability of reaching informative prefixes. Weak prefix control breaks this barrier, and once control is available, richer observations such as conditional sampling or logits can outperform top-$1$ access. Changing only the generator interface creates an exponential gap for KL-regularized outcome-reward post-training. Comments: Subjects: Machine Learning (cs.LG) Cite as: arXiv:2604.04855 [cs.LG]   (or arXiv:2604.04855v1 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2604.04855 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Amit Kiran Rege [view email] [v1] Mon, 6 Apr 2026 16:58:20 UTC (39 KB) Full-text links: Access Paper: View a PDF of the p...