[2503.22165] Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models

Computer Science > Machine Learning arXiv:2503.22165 (cs) [Submitted on 28 Mar 2025 (v1), last revised 3 Mar 2026 (this version, v4)] Title:Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models Authors:Zhanke Zhou, Zhaocheng Zhu, Xuan Li, Mikhail Galkin, Xiao Feng, Sanmi Koyejo, Jian Tang, Bo Han View a PDF of the paper titled Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models, by Zhanke Zhou and 7 other authors View PDF HTML (experimental) Abstract:Numerous applications of large language models (LLMs) rely on their ability to perform step-by-step reasoning. However, the reasoning behavior of LLMs remains poorly understood, posing challenges to research, development, and safety. To address this gap, we introduce landscape of thoughts (LoT), the first landscape visualization tool to inspect the reasoning trajectories with certain reasoning methods on any multi-choice dataset. We represent the textual states in a trajectory as numerical features that quantify the states' distances to the answer choices. These features are then visualized in two-dimensional plots using t-SNE. Qualitative and quantitative analysis with the landscape of thoughts effectively distinguishes between strong and weak models, correct and incorrect answers, as well as different reasoning tasks. It also uncovers undesirable reasoning patterns, such as low consistency and high uncertainty. Additionally, users can adapt LoT to a model that predict...