[2603.29805] From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability

Computer Science > Machine Learning arXiv:2603.29805 (cs) [Submitted on 31 Mar 2026] Title:From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability Authors:Max Hennick, Guillaume Corlouer View a PDF of the paper titled From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability, by Max Hennick and 1 other authors View PDF HTML (experimental) Abstract:A key problem in the modern study of AI is predicting and understanding emergent capabilities in models during training. Inspired by methods for studying reactions in quantum chemistry, we present the ``2-datapoint reduced density matrix". We show that this object provides a computationally efficient, unified observable of phase transitions during training. By tracking the eigenvalue statistics of the 2RDM over a sliding window, we derive two complementary signals: the spectral heat capacity, which we prove provides early warning of second-order phase transitions via critical slowing down, and the participation ratio, which reveals the dimensionality of the underlying reorganization. Remarkably, the top eigenvectors of the 2RDM are directly interpretable making it straightforward to study the nature of the transitions. We validate across four settings distinct settings: deep linear networks, induction head formation, grokking, and emergent misalignment. We then discuss directions for future work using the 2RDM. Subjects: Machine Lear...