![[2602.14862] The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.14862] The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling
Summary
The paper explores the properties of temperature scaling in probabilistic models, particularly its impact on classifier calibration and large language models, providing new theoretical insights and characterizations.
Why It Matters
Understanding temperature scaling is crucial for improving model calibration and performance in machine learning. This paper addresses gaps in theoretical analysis, offering insights that could enhance the application of temperature scaling in various AI contexts, including LLMs.
Key Takeaways
- Temperature scaling increases model uncertainty and entropy.
- The common belief that higher temperature increases diversity in LLMs is challenged.
- Introduces two new characterizations of temperature scaling, enhancing theoretical understanding.
Statistics > Machine Learning arXiv:2602.14862 (stat) [Submitted on 16 Feb 2026] Title:The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling Authors:Pierre-Alexandre Mattei, Bruno Loureiro View a PDF of the paper titled The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling, by Pierre-Alexandre Mattei and Bruno Loureiro View PDF HTML (experimental) Abstract:Temperature scaling is a simple method that allows to control the uncertainty of probabilistic models. It is mostly used in two contexts: improving the calibration of classifiers and tuning the stochasticity of large language models (LLMs). In both cases, temperature scaling is the most popular method for the job. Despite its popularity, a rigorous theoretical analysis of the properties of temperature scaling has remained elusive. We investigate here some of these properties. For classification, we show that increasing the temperature increases the uncertainty in the model in a very general sense (and in particular increases its entropy). However, for LLMs, we challenge the common claim that increasing temperature increases diversity. Furthermore, we introduce two new characterisations of temperature scaling. The first one is geometric: the tempered model is shown to be the information projection of the original model onto the set of models with a given entropy. The second characterisation clarifies the role of temperature scaling as a submodel of more general linear sc...
