![[2602.20404] $κ$-Explorer: A Unified Framework for Active Model Estimation in MDPs](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.20404] $κ$-Explorer: A Unified Framework for Active Model Estimation in MDPs
Summary
$κ$-Explorer presents a novel framework for active model estimation in Markov decision processes (MDPs), focusing on optimizing exploration strategies to improve model accuracy through a parameterized family of objective functions.
Why It Matters
This research is significant as it addresses the challenge of model estimation in MDPs by proposing a unified approach that enhances exploration efficiency. The findings could lead to advancements in reinforcement learning and decision-making systems, making it relevant for both academic and practical applications in machine learning.
Key Takeaways
- Introduces $κ$-Explorer, an active exploration algorithm for MDPs.
- Utilizes a parameterized family of objective functions to optimize exploration strategies.
- Demonstrates superior performance over existing exploration methods in benchmark tests.
- Establishes tight regret guarantees for the proposed algorithm.
- Offers a computationally efficient surrogate algorithm for practical applications.
Computer Science > Machine Learning arXiv:2602.20404 (cs) [Submitted on 23 Feb 2026] Title:$κ$-Explorer: A Unified Framework for Active Model Estimation in MDPs Authors:Xihe Gu, Urbashi Mitra, Tara Javidi View a PDF of the paper titled $\kappa$-Explorer: A Unified Framework for Active Model Estimation in MDPs, by Xihe Gu and 2 other authors View PDF Abstract:In tabular Markov decision processes (MDPs) with perfect state observability, each trajectory provides active samples from the transition distributions conditioned on state-action pairs. Consequently, accurate model estimation depends on how the exploration policy allocates visitation frequencies in accordance with the intrinsic complexity of each transition distribution. Building on recent work on coverage-based exploration, we introduce a parameterized family of decomposable and concave objective functions $U_\kappa$ that explicitly incorporate both intrinsic estimation complexity and extrinsic visitation frequency. Moreover, the curvature $\kappa$ provides a unified treatment of various global objectives, such as the average-case and worst-case estimation error objectives. Using the closed-form characterization of the gradient of $U_\kappa$, we propose $\kappa$-Explorer, an active exploration algorithm that performs Frank-Wolfe-style optimization over state-action occupancy measures. The diminishing-returns structure of $U_\kappa$ naturally prioritizes underexplored and high-variance transitions, while preserving sm...
