Memristive stochastic computing for deep learning parameter optimization

Lammie, Corey, Eshraghian, Jason K., Lu, Wei D., and Rahimi Azghadi, Mostafa (2021) Memristive stochastic computing for deep learning parameter optimization. IEEE Transactions on Circuits and Systems II: Express Briefs, 68 (5). pp. 1650-1654.

[img] PDF (Accepted Author Manuscript) - Accepted Version
Restricted to Repository staff only until 16 March 2023.

View at Publisher Website: https://doi.org/10.1109/TCSII.2021.30659...
 
6


Abstract

Stochastic Computing (SC) is a computing paradigm that allows for the low-cost and low-power computation of various arithmetic operations using stochastic bit streams and digital logic. In contrast to conventional representation schemes used within the binary domain, the sequence of bit streams in the stochastic domain is inconsequential, and computation is usually non-deterministic. In this brief, we exploit the stochasticity during switching of probabilistic Conductive Bridging RAM (CBRAM) devices to efficiently generate stochastic bit streams in order to perform Deep Learning (DL) parameter optimization, reducing the size of Multiply and Accumulate (MAC) units by 5 orders of magnitude. We demonstrate that in using a 40-nm Complementary Metal Oxide Semiconductor (CMOS) process our scalable architecture occupies 1.55mm 2 and consumes approximately 167 μW when optimizing parameters of a Convolutional Neural Network (CNN) while it is being trained for a character recognition task, observing no notable reduction in accuracy post-training.

Item ID: 68685
Item Type: Article (Research - C1)
ISSN: 1558-3791
Keywords: Stochastic processes; Training; Switches; Optimization; Performance evaluation; Computer architecture; Deep learning
Copyright Information: © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
Funders: James Cook University (JCU)
Projects and Grants: JCU DRTPS, JCU Rising Start ECR Fellowship
Date Deposited: 25 Jul 2021 21:48
FoR Codes: 40 ENGINEERING > 4018 Nanotechnology > 401804 Nanoelectronics @ 25%
40 ENGINEERING > 4009 Electronics, sensors and digital hardware > 400901 Analog electronics and interfaces @ 25%
46 INFORMATION AND COMPUTING SCIENCES > 4611 Machine learning > 461103 Deep learning @ 50%
SEO Codes: 28 EXPANDING KNOWLEDGE > 2801 Expanding knowledge > 280110 Expanding knowledge in engineering @ 100%
Downloads: Total: 6
Last 12 Months: 6
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page