![[2602.22070] Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.22070] Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts
Summary
This study explores how large language models (LLMs) exhibit inconsistent biases towards algorithmic agents and human experts in decision-making tasks, revealing significant implications for AI deployment.
Why It Matters
Understanding the biases of LLMs is crucial as they are increasingly used in decision-making roles. This research highlights the potential risks of deploying AI systems that may favor human experts over algorithms or vice versa, impacting trust and effectiveness in high-stakes scenarios.
Key Takeaways
- LLMs show higher trust ratings for human experts compared to algorithms.
- In incentivized betting scenarios, LLMs may favor algorithms even when they perform worse.
- The study underscores the importance of task presentation formats in evaluating AI biases.
- Inconsistent biases in LLMs can affect decision-making in critical applications.
- Robust scrutiny of LLM evaluation methods is essential for AI safety.
Computer Science > Artificial Intelligence arXiv:2602.22070 (cs) [Submitted on 25 Feb 2026] Title:Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts Authors:Jessica Y. Bo, Lillio Mok, Ashton Anderson View a PDF of the paper titled Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts, by Jessica Y. Bo and 2 other authors View PDF HTML (experimental) Abstract:Large language models are increasingly used in decision-making tasks that require them to process information from a variety of sources, including both human experts and other algorithmic agents. How do LLMs weigh the information provided by these different sources? We consider the well-studied phenomenon of algorithm aversion, in which human decision-makers exhibit bias against predictions from algorithms. Drawing upon experimental paradigms from behavioural economics, we evaluate how eightdifferent LLMs delegate decision-making tasks when the delegatee is framed as a human expert or an algorithmic agent. To be inclusive of different evaluation formats, we conduct our study with two task presentations: stated preferences, modeled through direct queries about trust towards either agent, and revealed preferences, modeled through providing in-context examples of the performance of both agents. When prompted to rate the trustworthiness of human experts and algorithms across diverse tasks, LLMs give higher ratings to the human expert, which correlates...
