![[2410.23029] Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learning](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2410.23029] Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learning
Summary
This paper explores risk-aware decision-making in restless bandits, proposing new algorithms for planning and learning that incorporate risk mitigation strategies.
Why It Matters
Understanding risk-aware decision-making is crucial for applications in fields like healthcare and resource management, where minimizing downside risks can significantly impact outcomes. This research advances the theory and practical algorithms for addressing these challenges in restless bandits.
Key Takeaways
- Introduces risk-aware objectives in the restless bandits framework.
- Establishes indexability conditions for risk-aware decision-making.
- Proposes a Thompson sampling approach for learning under uncertainty.
- Demonstrates the efficacy of the proposed methods through numerical experiments.
- Highlights applications in machine replacement and patient scheduling.
Computer Science > Machine Learning arXiv:2410.23029 (cs) [Submitted on 30 Oct 2024 (v1), last revised 19 Feb 2026 (this version, v3)] Title:Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learning Authors:Nima Akbarzadeh, Yossiri Adulyasak, Erick Delage View a PDF of the paper titled Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learning, by Nima Akbarzadeh and 2 other authors View PDF Abstract:In restless bandits, a central agent is tasked with optimally distributing limited resources across several bandits (arms), with each arm being a Markov decision process. In this work, we generalize the traditional restless bandits problem with a risk-neutral objective by incorporating risk-awareness, which is particularly important in various real-world applications especially when the decision maker seeks to mitigate downside risks. We establish indexability conditions for the case of a risk-aware objective and provide a solution based on Whittle index for the first time for the planning problem with finite-horizon non-stationary and for infinite-horizon stationary Markov decision processes. In addition, we address the learning problem when the true transition probabilities are unknown by proposing a Thompson sampling approach and show that it achieves bounded regret that scales sublinearly with the number of episodes and quadratically with the number of arms. The efficacy of our method in reducing ri...