[2504.14814] A Diagnostic Evaluation of Neural Networks Trained with the Error Diffusion Learning Algorithm

Computer Science > Machine Learning arXiv:2504.14814 (cs) [Submitted on 21 Apr 2025 (v1), last revised 2 Mar 2026 (this version, v4)] Title:A Diagnostic Evaluation of Neural Networks Trained with the Error Diffusion Learning Algorithm Authors:Kazuhisa Fujita View a PDF of the paper titled A Diagnostic Evaluation of Neural Networks Trained with the Error Diffusion Learning Algorithm, by Kazuhisa Fujita View PDF HTML (experimental) Abstract:The Error Diffusion Learning Algorithm (EDLA) is a learning scheme that performs synaptically local weight updates driven by a single, globally defined error signal. Although originally proposed as an alternative to backpropagation, its behavior has not been systematically characterized. We provide a modern formulation and implementation of EDLA and evaluate multilayer perceptrons trained with EDLA on parity, regression, and image-classification benchmarks (Digits, MNIST, Fashion-MNIST, and CIFAR-10). Following the original formulation, multi-class classification is implemented by training independent single-output networks (one per class), which makes the computational cost scale linearly with the number of classes. Under comparable architectures and training protocols, EDLA consistently underperforms backpropagation-trained baselines on all benchmarks considered. Through an analysis of internal dynamics, we identify a depth-related failure mode in ReLU-based EDLA: activations can grow explosively, causing unstable training and degraded ...