![[2506.03407] Multi-Spectral Gaussian Splatting with Neural Color Representation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2506.03407] Multi-Spectral Gaussian Splatting with Neural Color Representation
Summary
The paper presents MS-Splatting, a novel multi-spectral 3D Gaussian Splatting framework that generates consistent views from images captured by multiple cameras across different spectral domains, improving rendering quality without requiring cross-modal calibration.
Why It Matters
This research is significant as it addresses the limitations of existing multi-spectral rendering techniques by integrating various spectral domains into a unified model. The implications for fields like agriculture, where accurate spectral data is crucial for analyzing vegetation, could enhance remote sensing and environmental monitoring.
Key Takeaways
- MS-Splatting enables multi-view consistent rendering from diverse spectral data.
- The method eliminates the need for cross-modal camera calibration.
- It utilizes a neural color representation for improved spectral rendering quality.
- The approach is versatile, applicable to thermal and near-infrared imaging.
- Demonstrated effectiveness in agricultural applications, enhancing vegetation index analysis.
Computer Science > Graphics arXiv:2506.03407 (cs) [Submitted on 3 Jun 2025 (v1), last revised 16 Feb 2026 (this version, v2)] Title:Multi-Spectral Gaussian Splatting with Neural Color Representation Authors:Lukas Meyer, Josef Grün, Maximilian Weiherer, Bernhard Egger, Marc Stamminger, Linus Franke View a PDF of the paper titled Multi-Spectral Gaussian Splatting with Neural Color Representation, by Lukas Meyer and 5 other authors View PDF Abstract:We present MS-Splatting -- a multi-spectral 3D Gaussian Splatting (3DGS) framework that is able to generate multi-view consistent novel views from images of multiple, independent cameras with different spectral domains. In contrast to previous approaches, our method does not require cross-modal camera calibration and is versatile enough to model a variety of different spectra, including thermal and near-infra red, without any algorithmic changes. Unlike existing 3DGS-based frameworks that treat each modality separately (by optimizing per-channel spherical harmonics) and therefore fail to exploit the underlying spectral and spatial correlations, our method leverages a novel neural color representation that encodes multi-spectral information into a learned, compact, per-splat feature embedding. A shallow multi-layer perceptron (MLP) then decodes this embedding to obtain spectral color values, enabling joint learning of all bands within a unified representation. Our experiments show that this simple yet effective strategy is able to ...