[2603.01930] From Variance to Invariance: Qualitative Content Analysis for Narrative Graph Annotation

Computer Science > Computation and Language arXiv:2603.01930 (cs) [Submitted on 2 Mar 2026] Title:From Variance to Invariance: Qualitative Content Analysis for Narrative Graph Annotation Authors:Junbo Huang, Max Weinig, Ulrich Fritsche, Ricardo Usbeck View a PDF of the paper titled From Variance to Invariance: Qualitative Content Analysis for Narrative Graph Annotation, by Junbo Huang and 3 other authors View PDF HTML (experimental) Abstract:Narratives in news discourse play a critical role in shaping public understanding of economic events, such as inflation. Annotating and evaluating these narratives in a structured manner remains a key challenge for Natural Language Processing (NLP). In this work, we introduce a narrative graph annotation framework that integrates principles from qualitative content analysis (QCA) to prioritize annotation quality by reducing annotation errors. We present a dataset of inflation narratives annotated as directed acyclic graphs (DAGs), where nodes represent events and edges encode causal relations. To evaluate annotation quality, we employed a $6\times3$ factorial experimental design to examine the effects of narrative representation (six levels) and distance metric type (three levels) on inter-annotator agreement (Krippendorrf's $\alpha$), capturing the presence of human label variation (HLV) in narrative interpretations. Our analysis shows that (1) lenient metrics (overlap-based distance) overestimate reliability, and (2) locally-constrai...