![[2602.23280] Physics Informed Viscous Value Representations](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.23280] Physics Informed Viscous Value Representations
Summary
This paper presents a novel approach to offline goal-conditioned reinforcement learning by introducing a physics-informed regularization derived from the viscosity solution of the Hamilton-Jacobi-Bellman equation, improving value estimation in complex environments.
Why It Matters
Accurate value estimation in reinforcement learning is crucial for developing effective policies, especially in high-dimensional spaces. This research offers a new method that integrates physical principles, potentially enhancing the performance of algorithms in navigation and manipulation tasks.
Key Takeaways
- Introduces a physics-informed regularization for value estimation in reinforcement learning.
- Utilizes the viscosity solution of the Hamilton-Jacobi-Bellman equation for improved learning.
- Demonstrates enhanced geometric consistency in complex manipulation tasks.
- Applies the Feynman-Kac theorem for tractable Monte Carlo estimation.
- Provides open-source code for further research and application.
Computer Science > Machine Learning arXiv:2602.23280 (cs) [Submitted on 26 Feb 2026] Title:Physics Informed Viscous Value Representations Authors:Hrishikesh Viswanath, Juanwu Lu, S. Talha Bukhari, Damon Conover, Ziran Wang, Aniket Bera View a PDF of the paper titled Physics Informed Viscous Value Representations, by Hrishikesh Viswanath and 5 other authors View PDF HTML (experimental) Abstract:Offline goal-conditioned reinforcement learning (GCRL) learns goal-conditioned policies from static pre-collected datasets. However, accurate value estimation remains a challenge due to the limited coverage of the state-action space. Recent physics-informed approaches have sought to address this by imposing physical and geometric constraints on the value function through regularization defined over first-order partial differential equations (PDEs), such as the Eikonal equation. However, these formulations can often be ill-posed in complex, high-dimensional environments. In this work, we propose a physics-informed regularization derived from the viscosity solution of the Hamilton-Jacobi-Bellman (HJB) equation. By providing a physics-based inductive bias, our approach grounds the learning process in optimal control theory, explicitly regularizing and bounding updates during value iterations. Furthermore, we leverage the Feynman-Kac theorem to recast the PDE solution as an expectation, enabling a tractable Monte Carlo estimation of the objective that avoids numerical instability in high...