[2603.02366] PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR

Computer Science > Human-Computer Interaction arXiv:2603.02366 (cs) [Submitted on 2 Mar 2026] Title:PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR Authors:Esen K. Tütüncü, Qian Zhou, Frederik Brudy, George Fitzmaurice, Fraser Anderson View a PDF of the paper titled PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR, by Esen K. T\"ut\"unc\"u and 4 other authors View PDF HTML (experimental) Abstract:Current AI writing tools, which rely on text prompts, poorly support the spatial and interactive nature of storytelling where ideas emerge from direct manipulation and play. We present PlayWrite, a mixed-reality system where users author stories by directly manipulating virtual characters and props. A multi-agent AI pipeline interprets these actions into Intent Frames -structured narrative beats visualized as rearrangeable story marbles on a timeline. A large language model then transforms the user's assembled sequence into a final narrative. A user study (N=13) with writers from varying domains found that PlayWrite fosters a highly improvisational and playful process. Users treated the AI as a collaborative partner, using its unexpected responses to spark new ideas and overcome creative blocks. PlayWrite demonstrates an approach for co-creative systems that move beyond text to embrace direct manipulation and play as core interaction modalities. Subjects: Human-Computer Interaction (cs.HC); Artifici...