Approximate FPGA-based LSTMs under computation time constraints

Abstract

Recurrent Neural Networks, with the prominence of Long Short-Term Memory (LSTM) networks, have demonstrated state-of-the- art accuracy in several emerging Artificial Intelligence tasks. Neverthe- less, the highest performing LSTM models are becoming increasingly demanding in terms of computational and memory load. At the same time, emerging latency-sensitive applications including mobile robots and autonomous vehicles often operate under stringent computation time constraints. In this paper, we address the challenge of deploying com- putationally demanding LSTMs at a constrained time budget by intro- ducing an approximate computing scheme that combines iterative low- rank compression and pruning, along with a novel FPGA-based LSTM architecture. Combined in an end-to-end framework, the approximation method parameters are optimised and the architecture is configured to address the problem of high-performance LSTM execution in time- constrained applications. Quantitative evaluation on a real-life image captioning application indicates that the proposed system required up to 6.5 × less time to achieve the same application-level accuracy compared to a baseline method, while achieving an average of 25 × higher accuracy under the same computation time constraints.