![[2602.13891] GSRM: Generative Speech Reward Model for Speech RLHF](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13891] GSRM: Generative Speech Reward Model for Speech RLHF
Summary
The paper introduces the Generative Speech Reward Model (GSRM), a novel approach to evaluating speech naturalness in AI-generated audio, enhancing interpretability and performance in speech synthesis.
Why It Matters
As AI-generated speech becomes more prevalent, ensuring its naturalness is crucial for user experience. GSRM addresses limitations of existing evaluators by providing a more interpretable and effective method for assessing speech quality, which can significantly improve applications in voice assistants and other speech-related technologies.
Key Takeaways
- GSRM enhances speech naturalness evaluation through a two-stage process: feature extraction and reasoning.
- It is trained on a large dataset of human feedback, improving its predictive accuracy.
- GSRM outperforms existing models, achieving high correlation with human evaluations.
- The model can be integrated into online reinforcement learning from human feedback (RLHF) to refine speech generation.
- This advancement is significant for applications in AI voice synthesis and user interaction.
Computer Science > Sound arXiv:2602.13891 (cs) [Submitted on 14 Feb 2026] Title:GSRM: Generative Speech Reward Model for Speech RLHF Authors:Maohao Shen, Tejas Jayashankar, Osama Hanna, Naoyuki Kanda, Yancheng Wang, Kateřina Žmolíková, Ruiming Xie, Niko Moritz, Anfeng Xu, Yashesh Gaur, Gregory Wornell, Qing He, Jilong Wu View a PDF of the paper titled GSRM: Generative Speech Reward Model for Speech RLHF, by Maohao Shen and 12 other authors View PDF HTML (experimental) Abstract:Recent advances in speech language models, such as GPT-4o Voice Mode and Gemini Live, have demonstrated promising speech generation capabilities. Nevertheless, the aesthetic naturalness of the synthesized audio still lags behind that of human speech. Enhancing generation quality requires a reliable evaluator of speech naturalness. However, existing naturalness evaluators typically regress raw audio to scalar scores, offering limited interpretability of the evaluation and moreover fail to generalize to speech across different taxonomies. Inspired by recent advances in generative reward modeling, we propose the Generative Speech Reward Model (GSRM), a reasoning-centric reward model tailored for speech. The GSRM is trained to decompose speech naturalness evaluation into an interpretable acoustic feature extraction stage followed by feature-grounded chain-of-thought reasoning, enabling explainable judgments. To achieve this, we curated a large-scale human feedback dataset comprising 31k expert ratings an...
