[2603.21373] PLR: Plackett-Luce for Reordering In-Context Learning Examples

Computer Science > Machine Learning arXiv:2603.21373 (cs) [Submitted on 22 Mar 2026] Title:PLR: Plackett-Luce for Reordering In-Context Learning Examples Authors:Pawel Batorski, Paul Swoboda View a PDF of the paper titled PLR: Plackett-Luce for Reordering In-Context Learning Examples, by Pawel Batorski and 1 other authors View PDF HTML (experimental) Abstract:In-context learning (ICL) adapts large language models by conditioning on a small set of ICL examples, avoiding costly parameter updates. Among other factors, performance is often highly sensitive to the ordering of the examples. However, exhaustive search over the $n!$ possible orderings is infeasible. Therefore more efficient ordering methods use model confidence measures (e.g., label-probability entropy) over label sets or take a direct approach to finding the best ordering. We propose PLR, a probabilistic approach to in-context example ordering that replaces discrete ordering search with learning a probability distribution over orderings with the Plackett-Luce model. PLR models orderings using a Plackett-Luce distribution and iteratively updates its parameters to concentrate probability mass on high-performing orderings under a task-level metric. Candidate orderings are sampled efficiently via a Gumbel perturb-and-sort procedure. Experiments on multiple classification benchmarks show that PLR consistently improves few-shot accuracy for $k \in \{4, 8, 16, 32\}$ examples, and we further demonstrate gains on mathemat...