![[2602.15206] MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15206] MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference
Summary
The paper presents MAVRL, a novel approach for learning reward functions from multiple feedback types using amortized variational inference, enhancing robustness and interpretability in machine learning applications.
Why It Matters
Understanding how to effectively learn from diverse feedback types is crucial in machine learning, particularly in reinforcement learning. This research addresses a significant gap by proposing a unified method that improves performance and interpretability, which is essential for developing reliable AI systems.
Key Takeaways
- MAVRL formulates reward learning as Bayesian inference over a shared latent reward function.
- The method integrates multiple feedback types without the need for manual loss balancing.
- Jointly inferred reward posteriors show improved performance compared to single-type baselines.
- The approach enhances robustness to environmental changes and provides interpretable signals for model confidence.
- This research contributes to the advancement of AI systems that can learn from diverse and potentially conflicting feedback.
Computer Science > Machine Learning arXiv:2602.15206 (cs) [Submitted on 16 Feb 2026] Title:MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference Authors:Raphaël Baur, Yannick Metz, Maria Gkoulta, Mennatallah El-Assady, Giorgia Ramponi, Thomas Kleine Buening View a PDF of the paper titled MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference, by Rapha\"el Baur and 5 other authors View PDF HTML (experimental) Abstract:Reward learning typically relies on a single feedback type or combines multiple feedback types using manually weighted loss terms. Currently, it remains unclear how to jointly learn reward functions from heterogeneous feedback types such as demonstrations, comparisons, ratings, and stops that provide qualitatively different signals. We address this challenge by formulating reward learning from multiple feedback types as Bayesian inference over a shared latent reward function, where each feedback type contributes information through an explicit likelihood. We introduce a scalable amortized variational inference approach that learns a shared reward encoder and feedback-specific likelihood decoders and is trained by optimizing a single evidence lower bound. Our approach avoids reducing feedback to a common intermediate representation and eliminates the need for manual loss balancing. Across discrete and continuous-control benchmarks, we show that jointly inferred reward poste...
