[2603.22779] KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao

Computer Science > Information Retrieval arXiv:2603.22779 (cs) [Submitted on 24 Mar 2026] Title:KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao Authors:Zhi Sun, Wenming Zhang, Yi Wei, Liren Yu, Zhixuan Zhang, Dan Ou, Haihong Tang View a PDF of the paper titled KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao, by Zhi Sun and 6 other authors View PDF HTML (experimental) Abstract:Large Language Models (LLMs) are equipped with profound semantic knowledge, making them a natural choice for injecting semantic generalization into personalized search systems. However, in practice we find that directly fine-tuning LLMs on industrial personalized tasks (e.g. next item prediction) often yields suboptimal results. We attribute this bottleneck to a critical Knowledge--Action Gap: the inherent conflict between preserving pre-trained semantic knowledge and aligning with specific personalized actions by discriminative objectives. Empirically, action-only training objectives induce Semantic Collapse, such as attention ``sinks''. This degradation severely cripples the LLM's generalization, failing to bring improvements to personalized search systems. We propose KARMA (Knowledge--Action Regularized Multimodal Alignment), a unified framework that treats semantic reconstruction as a train-only regularizer. KARMA optimizes a next-interest embedding for retrieval (Action) while enforcing semantic decodability (Kn...