[2406.01969] Multiway Multislice PHATE: Visualizing Hidden Dynamics of RNNs through Training

Computer Science > Machine Learning arXiv:2406.01969 (cs) [Submitted on 4 Jun 2024 (v1), last revised 24 Mar 2026 (this version, v2)] Title:Multiway Multislice PHATE: Visualizing Hidden Dynamics of RNNs through Training Authors:Jiancheng Xie, Lou C. Kohler Voinov, Noga Mudrik, Gal Mishne, Adam Charles View a PDF of the paper titled Multiway Multislice PHATE: Visualizing Hidden Dynamics of RNNs through Training, by Jiancheng Xie and 4 other authors View PDF Abstract:Recurrent neural networks (RNNs) are a widely used tool for sequential data analysis; however, they are still often seen as black boxes. Visualizing the internal dynamics of RNNs is a critical step toward understanding their functional principles and developing better architectures and optimization strategies. Prior studies typically emphasize network representations only after training, overlooking how those representations evolve during learning. Here, we present Multiway Multislice PHATE (MM-PHATE), a graph-based embedding method for visualizing the evolution of RNN hidden states across the multiple dimensions spanned by RNNs: time, training epoch, and units. Across controlled synthetic benchmarks and real RNN applications, MM-PHATE preserves hidden-representation community structure among units and reveals training-phase changes in representation geometry. In controlled synthetic systems spanning multiple bifurcation families and smooth state-space warps, MM-PHATE recovers qualitative dynamical progression w...