Belson, Bruce, and Philippa, Bronson (2022) Speeding up Machine Learning Inference on Edge Devices by Improving Memory Access Patterns using Coroutines. In: Proceedings of the IEEE 25th International Conference on Computational Science and Engineering. pp. 9-16. From: CSE 2022: IEEE 25th International Conference on Computational Science and Engineering, 9-11 December 2022, Wuhan, China.

PDF (Published Version) - Published Version Restricted to Repository staff only

 

2

Abstract

We demonstrate a novel method of speeding up large iterative tasks such as machine learning inference. Our approach is to improve the memory access pattern, taking advantage of coroutines as a programming language feature to minimise the developer effort and reduce code complexity. We evaluate our approach using a comprehensive set of benchmarks run on three hardware platforms (one ARM and two Intel CPUs). The best observed performance boosts were 65% for scanning the nodes in a B+ tree, 34% for support vector machine inference, 12% for image pixel normalisation, and 15.5% for two dimensional convolution. Performance varied with data size, numeric type, and other factors, but overall the method is practical and can lead to significant improvements for edge computing.

Actions (Repository Staff Only)

Item Control Page