[2604.00830] Learning to Learn-at-Test-Time: Language Agents with Learnable Adaptation Policies

Computer Science > Machine Learning arXiv:2604.00830 (cs) [Submitted on 1 Apr 2026] Title:Learning to Learn-at-Test-Time: Language Agents with Learnable Adaptation Policies Authors:Zhanzhi Lou, Hui Chen, Yibo Li, Qian Wang, Bryan Hooi View a PDF of the paper titled Learning to Learn-at-Test-Time: Language Agents with Learnable Adaptation Policies, by Zhanzhi Lou and 4 other authors View PDF HTML (experimental) Abstract:Test-Time Learning (TTL) enables language agents to iteratively refine their performance through repeated interactions with the environment at inference time. At the core of TTL is an adaptation policy that updates the actor policy based on experience from previous episodes, thereby improving future behavior. Existing methods rely on fixed, hand-crafted adaptation policies rather than optimizing them for downstream improvement. We argue that optimal adaptation policies should be learned from task environments, not hand-engineered based on human intuition. To achieve this, we introduce Meta-TTL, a framework that formulates the discovery of effective adaptation policies as a bi-level optimization problem. Within this framework, the inner loop executes the standard TTL process, measuring how effectively a candidate adaptation policy helps an agent correct errors across sequential episodes. Guided by the agent's performance, the outer loop employs evolutionary search over a diverse distribution of training tasks to iteratively refine the adaptation policy. We ev...