![[2602.18671] Spilled Energy in Large Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.18671] Spilled Energy in Large Language Models
Summary
The paper explores the concept of 'spilled energy' in Large Language Models (LLMs), presenting a new method to detect factual errors and biases during decoding without additional training.
Why It Matters
Understanding energy dynamics in LLMs can enhance their reliability and performance, particularly in detecting hallucinations and biases, which is crucial for applications in AI safety and trustworthiness. This research contributes to ongoing efforts to improve the interpretability and accountability of AI systems.
Key Takeaways
- Introduces a novel approach to analyze LLMs using energy-based models.
- Demonstrates how 'spilled energy' correlates with factual inaccuracies and biases.
- Offers training-free metrics for hallucination detection in LLM outputs.
- Evaluated across multiple benchmarks, showing robust performance.
- Applicable to both pretrained and instruction-tuned LLMs without added training costs.
Computer Science > Artificial Intelligence arXiv:2602.18671 (cs) [Submitted on 21 Feb 2026] Title:Spilled Energy in Large Language Models Authors:Adrian Robert Minut, Hazem Dewidar, Iacopo Masi View a PDF of the paper titled Spilled Energy in Large Language Models, by Adrian Robert Minut and 2 other authors View PDF Abstract:We reinterpret the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interacting EBMs at inference. This principled approach allows us to track "energy spills" during decoding, which we empirically show correlate with factual errors, biases, and failures. Similar to Orgad et al. (2025), our method localizes the exact answer token and subsequently tests for hallucinations. Crucially, however, we achieve this without requiring trained probe classifiers or activation ablations. Instead, we introduce two completely training-free metrics derived directly from output logits: spilled energy, which captures the discrepancy between energy values across consecutive generation steps that should theoretically match, and marginalized energy, which is measurable at a single step. Evaluated on nine benchmarks across state-of-the-art LLMs (including LLaMA, Mistral, and Gemma) and on synthetic algebraic operations (Qwen3), our approach demonstrates robust, competitive hallucination detection and cross-task generalization. Notably, these results hold for both pretraine...
