![[2602.20323] Learning Physical Principles from Interaction: Self-Evolving Planning via Test-Time Memory](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.20323] Learning Physical Principles from Interaction: Self-Evolving Planning via Test-Time Memory
Summary
This article presents PhysMem, a memory framework that allows vision-language model planners to learn physical principles through interaction, enhancing object manipulation tasks without needing to update model parameters.
Why It Matters
Understanding physical properties is crucial for effective object manipulation in robotics. PhysMem's approach to learning from interaction at test time represents a significant advancement in how robots can adapt to varying physical conditions, making them more versatile in real-world applications.
Key Takeaways
- PhysMem enables robots to learn physical principles from interactions without model updates.
- The system improves decision-making by verifying hypotheses against new observations.
- Real-world tests show a significant success rate improvement over traditional methods.
Computer Science > Robotics arXiv:2602.20323 (cs) [Submitted on 23 Feb 2026] Title:Learning Physical Principles from Interaction: Self-Evolving Planning via Test-Time Memory Authors:Haoyang Li, Yang You, Hao Su, Leonidas Guibas View a PDF of the paper titled Learning Physical Principles from Interaction: Self-Evolving Planning via Test-Time Memory, by Haoyang Li and 3 other authors View PDF HTML (experimental) Abstract:Reliable object manipulation requires understanding physical properties that vary across objects and environments. Vision-language model (VLM) planners can reason about friction and stability in general terms; however, they often cannot predict how a specific ball will roll on a particular surface or which stone will provide a stable foundation without direct experience. We present PhysMem, a memory framework that enables VLM robot planners to learn physical principles from interaction at test time, without updating model parameters. The system records experiences, generates candidate hypotheses, and verifies them through targeted interaction before promoting validated knowledge to guide future decisions. A central design choice is verification before application: the system tests hypotheses against new observations rather than applying retrieved experience directly, reducing rigid reliance on prior experience when physical conditions change. We evaluate PhysMem on three real-world manipulation tasks and simulation benchmarks across four VLM backbones. On a ...