![[2602.15634] Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15634] Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
Summary
This paper explores the limitations of Graph Neural Networks (GNNs) due to oversmoothing and proposes a novel approach using bifurcation theory to enhance node representation stability.
Why It Matters
The findings address a critical challenge in machine learning, particularly in GNNs, where oversmoothing leads to loss of informative features. By introducing a new activation function approach, this research could significantly improve the performance and applicability of GNNs in various domains.
Key Takeaways
- Oversmoothing in GNNs leads to convergence to a non-informative state.
- Bifurcation theory provides a new perspective on stabilizing GNN representations.
- Replacing standard activations like ReLU can create stable, non-homogeneous patterns.
- The theory predicts a scaling law for emergent patterns, validated through experiments.
- A bifurcation-aware initialization method enhances GNN performance in benchmarks.
Computer Science > Machine Learning arXiv:2602.15634 (cs) [Submitted on 17 Feb 2026] Title:Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors Authors:Erkan Turan, Gaspard Abel, Maysam Behmanesh, Emery Pierson, Maks Ovsjanikov View a PDF of the paper titled Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors, by Erkan Turan and 4 other authors View PDF HTML (experimental) Abstract:Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a \emph{bifurcation theory} perspective, characterizing oversmoothing as convergence to a stable ``homogeneous fixed point.'' Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous \emph{patterns} that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-awar...