![[2407.15738] Parallel Split Learning with Global Sampling](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2407.15738] Parallel Split Learning with Global Sampling
Summary
The paper presents a novel server-driven sampling strategy for distributed deep learning, enhancing scalability and accuracy in resource-constrained environments.
Why It Matters
This research addresses significant challenges in distributed deep learning, particularly in scenarios with limited resources and non-uniform data distribution. By improving model accuracy and training efficiency, it has implications for edge computing and real-world applications where data is decentralized.
Key Takeaways
- Introduces a server-driven sampling strategy for distributed learning.
- Maintains fixed global batch size while adjusting client-side batch sizes.
- Improves model accuracy and convergence stability.
- Provides tighter deviation guarantees using concentration bounds.
- Offers a scalable solution for learning in resource-constrained environments.
Computer Science > Machine Learning arXiv:2407.15738 (cs) [Submitted on 22 Jul 2024 (v1), last revised 25 Feb 2026 (this version, v5)] Title:Parallel Split Learning with Global Sampling Authors:Mohammad Kohankhaki, Ahmad Ayad, Mahdi Barhoush, Anke Schmeink View a PDF of the paper titled Parallel Split Learning with Global Sampling, by Mohammad Kohankhaki and 3 other authors View PDF HTML (experimental) Abstract:Distributed deep learning in resource-constrained environments faces scalability and generalization challenges due to large effective batch sizes and non-identically distributed client data. We introduce a server-driven sampling strategy that maintains a fixed global batch size by dynamically adjusting client-side batch sizes. This decouples the effective batch size from the number of participating devices and ensures that global batches better reflect the overall data distribution. Using standard concentration bounds, we establish tighter deviation guarantees compared to existing approaches. Empirical results on a benchmark dataset confirm that the proposed method improves model accuracy, training efficiency, and convergence stability, offering a scalable solution for learning at the network edge. Comments: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) Cite as: arXiv:2407.15738 [cs.LG] (or arXiv:2407.15738v5 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2407.15738 Focus to le...
