Structure Optimization in Deep Neural Networks with Synaptic Pruning Based on Connection Appraisal

Computer and Knowledge Engineering

مقالات آماده انتشار، پذیرفته شده، انتشار آنلاین از تاریخ 07 تیر 1404   

نوع مقاله: Machine learning-Sadoghi

شناسه دیجیتال (DOI): 10.22067/cke.2025.88039.1113

نویسندگان

Aghil Ahmadi¹؛ Reza Mahboobi Esfanjani^{* 2}

¹Department of Electrical Engineering, Sahand University of Technology, Tabriz, Iran

²Department of Electrical Engineering, Sahand University of Technology

چکیده

Deep neural networks typically require predefined architectures, which can lead to overfitting, underfitting, high computational costs, and storage overhead. Dynamic structure optimization through pruning can reduce network redundancy, but it often results in performance degradation. In this study, we propose a novel pruning method inspired by biological synaptic pruning that adaptively optimizes deep neural network structures. The proposed method continuously monitors the contribution of each connection during training using a dynamic efficiency criterion that evaluates the relative importance of each connection within its layer. Connections are not removed immediately; only those consistently falling below a predefined threshold are pruned, ensuring stability and robustness. Validation is conducted by simulation on an industrial distillation column dataset under noisy conditions and the MNIST benchmark dataset. The results demonstrate improved accuracy, enhanced generalization, and faster learning, with an average pruning rate of 53%. Compared to conventional and state-of-the-art pruning techniques, our method achieves superior performance in terms of compression rate and accuracy while effectively mitigating overfitting.

کلیدواژه‌ها

Deep Neural Networks؛ Synaptic Pruning؛ Distillation Column؛ Connection Evaluation