Cai, Zhengyu, Kalatehbali, Hamid Rahimian, Walters, Ben, Rahimi Azghadi, Mostafa, Amirsoleimani, Amirali, and Genov, Roman (2023) Spike Timing Dependent Gradient for Direct Training of Fast and Efficient Binarized Spiking Neural Networks. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 13 (4). pp. 1083-1093.

PDF (Published Version) - Published Version Restricted to Repository staff only

Abstract

Spiking neural networks (SNNs) are well-suited for neuromorphic hardware due to their biological plausibility and energy efficiency. These networks utilize sparse, asynchronous spikes for communication and can be binarized. However, the training of such networks presents several challenges due to their non-differentiable activation function and binarized inter-layer data movement. The well-established backpropagation through time (BPTT) algorithm used to train SNNs encounters notable difficulties because of its substantial memory consumption and extensive computational demands. These limitations restrict its practical utility in real-world scenarios. Therefore, effective techniques are required to train such networks efficiently while preserving accuracy. In this paper, we propose Binarized Spike Timing Dependent Gradient (BSTDG), a novel method that utilizes presynaptic and postsynaptic timings to bypass the non-differentiable gradient and the need of BPTT. Additionally, we employ binarized weights with a threshold training strategy to enhance energy savings and performance. Moreover, we exploit latency/temporal-based coding and the Integrate-and-Fire (IF) model to achieve significant computational advantages. We evaluate the proposed method on Caltech101 Face/Motorcycle, MNIST, Fashion-MNIST, and Spiking Heidelberg Digits. The results demonstrate that the accuracy attained surpasses that of existing BSNNs and single-spike networks under the same structure. Furthermore, the proposed model achieves up to 30 ××× speedup in inference and effectively reduces the number of spikes emitted in the hidden layer by 50% compared to previous works.

Item ID:	81670
Item Type:	Article (Research - C1)
ISSN:	2156-3365
Copyright Information:	© 2023 IEEE.
Date Deposited:	23 Jan 2024 23:44
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460207 Modelling and simulation @ 50% 46 INFORMATION AND COMPUTING SCIENCES > 4611 Machine learning > 461104 Neural networks @ 50%
SEO Codes:	22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220403 Artificial intelligence @ 100%
Downloads:	Total: 1 Last 12 Months: 1
	More Statistics