Stochastic computing for low-power and high-speed deep learning on FPGA

Lammie, Corey, and Azghadi, Mostafa Rahimi (2019) Stochastic computing for low-power and high-speed deep learning on FPGA. In: Proceedings of the IEEE International Symposium on Circuits and Systems. From: ISCAS 2019: IEEE International Symposium on Circuits and Systems, 26-29 May 2019, Sapporo, Japan.

[img] PDF (Published Version) - Published Version
Restricted to Repository staff only

View at Publisher Website: https://doi.org/10.1109/ISCAS.2019.87022...
 
1
2


Abstract

Stochastic Computing (SC) presents a low-cost and low-power alternative to conventional binary computing. In SC, continuous values are represented by stochastically generated bit streams. By performing simple hardware-friendly bit-wise operations on these streams, complex calculations can be realized very efficiently. However, the inherent randomness and approximation used in SC can result in undesirable computational errors. As Convolutional Neural Networks (CNNs) are inherently error-tolerant, SC could be embedded in them to gain higher speed and lower power without significant accuracy loss. In this paper, we propose using SC techniques to approximate multiplication operations on fixed-point weights and biases during training of CNNs. By employing such techniques, we demonstrate near state-of-the-art learning performance for the MNIST and CIFAR-10 datasets, while achieving significant resource and speed improvements when implementing the deep networks on a Field Programmable Gate Array (FPGA). For MNIST, we demonstrate that SC compared to conventional computing, will result in almost 3 times increase in learning speed with only 1.37% degradation in validation accuracy. Similarly, for CIFAR-10, training is accelerated 3.5 times with a degradation of 3.39%. We also show that our FPGA implementations of CNNs adopting stochastic multipliers consume over 17 times less power than their GPU counterparts.

Item ID: 61912
Item Type: Conference Item (Research - E1)
ISBN: 978-1-7281-0397-6
Keywords: CIFAR-10, CNN, Deep Learning, FPGA, High-Speed, Low-Power, MNIST, Stochastic Computing
Copyright Information: © 2019 IEEE
Date Deposited: 03 Feb 2020 00:50
FoR Codes: 09 ENGINEERING > 0906 Electrical and Electronic Engineering > 090601 Circuits and Systems @ 50%
10 TECHNOLOGY > 1006 Computer Hardware > 100601 Arithmetic and Logic Structures @ 50%
SEO Codes: 97 EXPANDING KNOWLEDGE > 970109 Expanding Knowledge in Engineering @ 100%
Downloads: Total: 2
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page