ALTK‑Evolve: On‑the‑Job Learning for AI Agents

Back to Articles ALTK‑Evolve: On‑the‑Job Learning for AI Agents Enterprise Article Published April 8, 2026 Upvote 8 +2 Vatche Isahagian Vatche Follow ibm-research Vinod Muthusamy vinodmut Follow ibm-research Jayaram Radhakrishnan jayaramkr Follow ibm-research Gaodan Fang gaodan-fang Follow ibm-research Punleuk Oum illeatmyhat Follow ibm-research G Thomas gsthomasx Follow ibm-research TL;DR Most AI agents re‑read transcripts instead of learning principles, so they repeat mistakes and don’t transfer lessons to new situations. ALTK‑Evolve turns raw agent trajectories into reusable guidelines. In benchmarks, the approach boosted reliability, especially on hard (Δ 14.2% on AppWorld), multi‑step tasks, without bloating context. The “eternal intern” problem Imagine a brilliant line cook who has memorized every cookbook but forgets your kitchen every morning. They don’t remember your oven runs hot, or that regulars like extra salt; they’ll follow a recipe card yet freeze when you’re out of lemons. That’s most AI agents: excellent at following prompts, poor at accumulating wisdom about your environment. Feeding yesterday’s logs back into the prompt just makes them re‑read history; it doesn’t help them generalize from it. A junior needs different recipes for “vinaigrette” and “duck à l’orange.” A chef learns “acid balances fat” and applies it everywhere. Likewise, reliable agents should distill principles from experience and apply them to new tasks, not just near duplicates of old o...