[2603.04257] Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory

Computer Science > Computation and Language arXiv:2603.04257 (cs) [Submitted on 4 Mar 2026] Title:Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory Authors:Zhenting Wang, Huancheng Chen, Jiayun Wang, Wei Wei View a PDF of the paper titled Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory, by Zhenting Wang and 3 other authors View PDF HTML (experimental) Abstract:Large language model (LLM) agents are fundamentally bottlenecked by finite context windows on long-horizon tasks. As trajectories grow, retaining tool outputs and intermediate reasoning in-context quickly becomes infeasible: the working context becomes prohibitively long, eventually exceeds the context budget, and makes distant evidence harder to use even when it is still present. Existing solutions typically shorten context through truncation or running summaries, but these methods are fundamentally lossy because they compress or discard past evidence itself. We introduce Memex, an indexed experience memory mechanism that instead compresses context without discarding evidence. Memex maintains a compact working context consisting of concise structured summaries and stable indices, while storing full-fidelity underlying interactions in an external experience database under those indices. The agent can then decide when to dereference an index and recover the exact past evidence needed for the current subgoal. We optimize both write and read behaviors with our reinforcemen...