[2604.00258] Hierarchical Apprenticeship Learning from Imperfect Demonstrations with Evolving Rewards

Computer Science > Machine Learning arXiv:2604.00258 (cs) [Submitted on 31 Mar 2026] Title:Hierarchical Apprenticeship Learning from Imperfect Demonstrations with Evolving Rewards Authors:Md Mirajul Islam, Rajesh Debnath, Adittya Soukarjya Saha, Min Chi View a PDF of the paper titled Hierarchical Apprenticeship Learning from Imperfect Demonstrations with Evolving Rewards, by Md Mirajul Islam and 3 other authors View PDF HTML (experimental) Abstract:While apprenticeship learning has shown promise for inducing effective pedagogical policies directly from student interactions in e-learning environments, most existing approaches rely on optimal or near-optimal expert demonstrations under a fixed reward. Real-world student interactions, however, are often inherently imperfect and evolving: students explore, make errors, revise strategies, and refine their goals as understanding develops. In this work, we argue that imperfect student demonstrations are not noise to be discarded, but structured signals-provided their relative quality is ranked. We introduce HALIDE, Hierarchical Apprenticeship Learning from Imperfect Demonstrations with Evolving Rewards, which not only leverages sub-optimal student demonstrations, but ranks them within a hierarchical learning framework. HALIDE models student behavior at multiple levels of abstraction, enabling inference of higher-level intent and strategy from suboptimal actions while explicitly capturing the temporal evolution of student reward f...