![[2602.21965] Compact Circulant Layers with Spectral Priors](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.21965] Compact Circulant Layers with Spectral Priors
Summary
This paper explores compact circulant layers with spectral priors, focusing on their application in memory-efficient neural networks for resource-constrained environments.
Why It Matters
As machine learning applications expand into areas like medicine and robotics, the need for efficient neural networks that are both compact and uncertainty-aware becomes critical. This research presents innovative methods to enhance neural network performance while reducing resource consumption, making it relevant for developers and researchers in the field.
Key Takeaways
- Compact spectral circulant layers can significantly reduce the number of parameters in neural networks.
- The proposed methods enable structured variational inference, improving robustness diagnostics.
- Empirical results show that these layers perform comparably to strong baselines while being more resource-efficient.
Computer Science > Machine Learning arXiv:2602.21965 (cs) [Submitted on 25 Feb 2026] Title:Compact Circulant Layers with Spectral Priors Authors:Joseph Margaryan, Thomas Hamelryck View a PDF of the paper titled Compact Circulant Layers with Spectral Priors, by Joseph Margaryan and Thomas Hamelryck View PDF HTML (experimental) Abstract:Critical applications in areas such as medicine, robotics and autonomous systems require compact (i.e., memory efficient), uncertainty-aware neural networks suitable for edge and other resource-constrained deployments. We study compact spectral circulant and block-circulant-with-circulant-blocks (BCCB) layers: FFT-diagonalizable circular convolutions whose weights live directly in the real FFT (RFFT) half (1D) or half-plane (2D). Parameterizing filters in the frequency domain lets us impose simple spectral structure, perform structured variational inference in a low-dimensional weight space, and calculate exact layer spectral norms, enabling inexpensive global Lipschitz bounds and margin-based robustness diagnostics. By placing independent complex Gaussians on the Hermitian support we obtain a discrete instance of the spectral representation of stationary kernels, inducing an exact stationary Gaussian-process prior over filters on the discrete circle/torus. We exploit this to define a practical spectral prior and a Hermitian-aware low-rank-plus-diagonal variational posterior in real coordinates. Empirically, spectral circulant/BCCB layers are...