Dong, Xuening, Chen, Brian, Genov, Roman, Rahimi Azghadi, Mostafa, and Amirsoleimani, Amirali (2024) SITU: Stochastic input encoding and weight update thresholding for efficient memristive neural network in-situ training. Neurocomputing, 605. 128275.

PDF (Published Version) - Published Version Restricted to Repository staff only

Abstract

The Analog-to-Digital Converter (ADC) sensing and conductance update are the most power-demanding processes in the in-situ training of memristive neural networks. In this work, we propose a new thresholded weight update method in conjunction with stochastic input encoding to reduce the ADC sensing requirement and weight updates. This leads to better power and area efficiency for simple neural network models developed in situ on memristive crossbars. For performance analysis, we train three different neural network architectures, i.e. two Convolutional Neural Networks (CNNs) and one Recurrent Neural Network (RNN) on our in-situ training simulation platform with the MNIST, CIFAR-10, and Sentimental Analysis datasets, using a passive crossbar array. Results show that as the network complexity increases, the performance of the in-situ trained network with our proposed update rules decreases, due to the reduction in the number of updates at the beginning of the training process and the problem of deciding the learning rate without prior knowledge of the gradient of the loss function. Compared to a baseline network, the proposed adaptive thresholding saves 52.0% and 59.1% of weight updates on CNN and RNN, by sacrificing 0.17% and 5.81% of classification accuracy, respectively. Moreover, the stochastic input encoding saves inference power by 4.97% and 3.48% on CNN and RNN, while also slightly reducing the layout area by around 5 x 10 -3 and 10 -3 mm2, respectively.

Item ID:	85307
Item Type:	Article (Research - C1)
ISSN:	1872-8286
Copyright Information:	© 2024 Elsevier B.V. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
Date Deposited:	01 May 2025 01:28
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4611 Machine learning > 461104 Neural networks @ 50% 40 ENGINEERING > 4008 Electrical engineering > 400802 Electrical circuits and systems @ 50%
SEO Codes:	22 INFORMATION AND COMMUNICATION SERVICES > 2299 Other information and communication services > 229999 Other information and communication services not elsewhere classified @ 100%
	More Statistics