[2605.07379] RELO: Reinforcement Learning to Localize for Visual Object Tracking

Computer Science > Computer Vision and Pattern Recognition arXiv:2605.07379 (cs) [Submitted on 8 May 2026] Title:RELO: Reinforcement Learning to Localize for Visual Object Tracking Authors:Xin Chen, Chuanyu Sun, Jiao Xu, Houwen Peng, Dong Wang, Huchuan Lu, Kede Ma View a PDF of the paper titled RELO: Reinforcement Learning to Localize for Visual Object Tracking, by Xin Chen and 6 other authors View PDF HTML (experimental) Abstract:Conventional visual object trackers localize targets using handcrafted spatial priors, often in the form of heatmaps. Such priors provide only surrogate supervision and are poorly aligned with tracking optimization and evaluation metrics, such as intersection over union (IoU) and area under the success curve (AUC). Here, we introduce RELO, a REinforcement-learning-to-LOcalize method for visual object tracking that formulates target localization as a Markov decision process. Specifically, RELO replaces handcrafted spatial priors with a localization policy learned over spatial positions via reinforcement learning, with rewards combining frame-level IoU and sequence-level AUC. We additionally introduce layer-aligned temporal token propagation to improve semantic consistency across frames, with negligible computational overhead. Across multiple benchmarks, RELO achieves superior results, attaining 57.5% AUC on LaSOText without template updates. This confirms that reward-driven localization provides an effective alternative to prior-driven localizatio...