[2505.02872] Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading

Computer Science > Computation and Language arXiv:2505.02872 (cs) [Submitted on 4 May 2025 (v1), last revised 2 Mar 2026 (this version, v3)] Title:Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading Authors:Cfir Avraham Hadar, Omer Shubi, Yoav Meiri, Amit Heshes, Yevgeni Berzak View a PDF of the paper titled Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading, by Cfir Avraham Hadar and 4 other authors View PDF HTML (experimental) Abstract:When reading, we often have specific information that interests us in a text. For example, you might be reading this paper because you are curious about LLMs for eye movements in reading, the experimental design, or perhaps you wonder ``This sounds like science fiction. Does it actually work?''. More broadly, in daily life, people approach texts with any number of text-specific goals that guide their reading behavior. In this work, we ask, for the first time, whether open-ended reading goals can be automatically decoded solely from eye movements in reading. To address this question, we introduce goal decoding tasks and evaluation frameworks using large-scale eye tracking for reading data in English with hundreds of text-specific information seeking tasks. We develop and compare several discriminative and generative multimodal text and eye movements LLMs for these tasks. Our experiments show considerable success on the task of selecting the correct goal among several options, and even progres...