[2410.05254] GLEE: A Unified Framework and Benchmark for Language-based Economic Environments

Computer Science > Computation and Language arXiv:2410.05254 (cs) [Submitted on 7 Oct 2024 (v1), last revised 2 Mar 2026 (this version, v3)] Title:GLEE: A Unified Framework and Benchmark for Language-based Economic Environments Authors:Eilam Shapira, Omer Madmon, Itamar Reinman, Samuel Joseph Amouyal, Roi Reichart, Moshe Tennenholtz View a PDF of the paper titled GLEE: A Unified Framework and Benchmark for Language-based Economic Environments, by Eilam Shapira and 5 other authors View PDF HTML (experimental) Abstract:Large Language Models (LLMs) show significant potential in economic and strategic interactions, where communication via natural language is often prevalent. This raises key questions: Do LLMs behave rationally? How do they perform compared to humans? Do they tend to reach an efficient and fair outcome? What is the role of natural language in strategic interaction? How do characteristics of the economic environment influence these dynamics? These questions become crucial concerning the economic and societal implications of integrating LLM-based agents into real-world data-driven systems, such as online retail platforms and recommender systems. To answer these questions, we introduce a benchmark for standardizing research on two-player, sequential, language-based games. Inspired by the economic literature, we define three base families of games with consistent parameterization, degrees of freedom and economic measures to evaluate agents' performance (self-gain),...