[2603.29888] Generative AI in Action: Field Experimental Evidence from Alibaba's Customer Service Operations

Computer Science > Human-Computer Interaction arXiv:2603.29888 (cs) [Submitted on 8 Feb 2026] Title:Generative AI in Action: Field Experimental Evidence from Alibaba's Customer Service Operations Authors:Xiao Ni, Yiwei Wang, Tianjun Feng, Lauren Xiaoyan Lu, Yitong Wang, Congyi Zhou View a PDF of the paper titled Generative AI in Action: Field Experimental Evidence from Alibaba's Customer Service Operations, by Xiao Ni and 5 other authors View PDF Abstract:In collaboration with Alibaba, this study leverages a large-scale field experiment to assess the impact of a generative AI assistant on worker performance in e-commerce after-sales service. Human agents providing digital chat support were randomly assigned with access to a gen AI assistant that offered two core functions: diagnosis of customer issues and solution proposals, presented as text messages. Agents retained discretion to adopt, modify, or disregard AI-generated messages. To evaluate gen AI's impact, we estimate both the intention-to-treat (ITT) effect of gen AI access and the local average treatment effect (LATE) of gen AI usage. Results show that gen AI significantly improved service speed, measured by issue identification time and chat duration. Gen AI also improved subjective service quality reflected in customer ratings and dissatisfaction rates, but it had no significant effect on objective service quality indicated by customer retrial rates. The performance improvements stemmed not only from automation but...