[2603.26840] Dual-branch Graph Domain Adaptation for Cross-scenario Multi-modal Emotion Recognition

Electrical Engineering and Systems Science > Audio and Speech Processing arXiv:2603.26840 (eess) [Submitted on 27 Mar 2026] Title:Dual-branch Graph Domain Adaptation for Cross-scenario Multi-modal Emotion Recognition Authors:Yuntao Shou, Jun Zhou, Tao Meng, Wei Ai, Keqin Li View a PDF of the paper titled Dual-branch Graph Domain Adaptation for Cross-scenario Multi-modal Emotion Recognition, by Yuntao Shou and 4 other authors View PDF HTML (experimental) Abstract:Multimodal Emotion Recognition in Conversations (MERC) aims to predict speakers' emotional states in multi-turn dialogues through text, audio, and visual cues. In real-world settings, conversation scenarios differ significantly in speakers, topics, styles, and noise levels. Existing MERC methods generally neglect these cross-scenario variations, limiting their ability to transfer models trained on a source domain to unseen target domains. To address this issue, we propose a Dual-branch Graph Domain Adaptation framework (DGDA) for multimodal emotion recognition under cross-scenario conditions. We first construct an emotion interaction graph to characterize complex emotional dependencies among utterances. A dual-branch encoder, consisting of a hypergraph neural network (HGNN) and a path neural network (PathNN), is then designed to explicitly model multivariate relationships and implicitly capture global dependencies. To enable out-of-domain generalization, a domain adversarial discriminator is introduced to learn inv...