![[2502.00944] Training speedups via batching for geometric learning: an analysis of static and dynamic algorithms](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2502.00944] Training speedups via batching for geometric learning: an analysis of static and dynamic algorithms
Summary
This article analyzes the impact of static and dynamic batching algorithms on training speed and performance in graph neural networks (GNNs), revealing potential speedups of up to 2.7x depending on various factors.
Why It Matters
As GNNs gain traction in fields like materials science and social sciences, optimizing their training processes becomes crucial. This research provides insights into how different batching strategies can enhance efficiency, which is vital for researchers and practitioners looking to improve model performance and reduce training times.
Key Takeaways
- Static and dynamic batching algorithms can significantly affect GNN training speed.
- Up to a 2.7x speedup is achievable, depending on data and model specifics.
- The choice of batching algorithm impacts model learning metrics.
- Understanding these algorithms can help optimize GNN applications in various domains.
- Experimentation is key to finding the best batching strategy for specific use cases.
Computer Science > Machine Learning arXiv:2502.00944 (cs) [Submitted on 2 Feb 2025 (v1), last revised 24 Feb 2026 (this version, v4)] Title:Training speedups via batching for geometric learning: an analysis of static and dynamic algorithms Authors:Daniel T. Speckhard, Tim Bechtel, Sebastian Kehl, Jonathan Godwin, Claudia Draxl View a PDF of the paper titled Training speedups via batching for geometric learning: an analysis of static and dynamic algorithms, by Daniel T. Speckhard and 4 other authors View PDF HTML (experimental) Abstract:Graph neural networks (GNN) have shown promising results for several domains such as materials science, chemistry, and the social sciences. GNN models often contain millions of parameters, and like other neural network (NN) models, are often fed only a fraction of the graphs that make up the training dataset in batches to update model parameters. The effect of batching algorithms on training time and model performance has been thoroughly explored for NNs but not yet for GNNs. We analyze two different batching algorithms for graph-based models, namely static and dynamic batching for two datasets, the QM9 dataset of small molecules and the AFLOW materials database. Our experiments show that changing the batching algorithm can provide up to a 2.7x speedup, but the fastest algorithm depends on the data, model, batch size, hardware, and number of training steps run. Experiments show that for a select number of combinations of batch size, dataset,...
