ATHEENA: a toolflow for hardware early-exit network automation
File(s)BiggsFCCM2023.pdf (676.8 KB)
Accepted version
Author(s)
Biggs, Benjamin
Bouganis, Christos-Savvas
Constantinides, George
Type
Conference Paper
Abstract
The continued need for improvements in accuracy, throughput, and efficiency of Deep Neural Networks has resulted in a multitude of methods that make the most of custom architectures on FPGAs. These include the creation of hand-crafted networks and the use of quantization and pruning to reduce extraneous network parameters. However, with the potential of static solutions already well exploited, we propose to shift the focus to using the varying difficulty of individual data samples to further improve efficiency and reduce average compute for classification. Input-dependent computation allows for the network to make runtime decisions to finish a task early if the result meets a confidence threshold. Early-Exit network architectures have become an increasingly popular way to implement such behaviour in software. We create A Toolflow for Hardware Early-Exit Network Automation (ATHEENA), an automated FPGA toolflow that leverages the probability of samples exiting early from such networks to scale the resources allocated to different sections of the network. The toolflow uses the data-flow model of fpgaConvNet, extended to support Early-Exit networks as well as Design Space Exploration to optimize the generated streaming architecture hardware with the goal of increasing throughput/reducing area while maintaining accuracy. Experimental results on three different networks demonstrate a throughput increase of 2.00× to 2.78× compared to an optimized baseline network implementation with no early exits. Additionally, the toolflow can achieve a throughput matching the same baseline with as low as 46% of the resources the baseline requires.
Date Issued
2023-07-10
Date Acceptance
2023-03-17
Citation
2023 IEEE 31st Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), 2023, pp.121-132
ISSN
2576-2621
Publisher
IEEE
Start Page
121
End Page
132
Journal / Book Title
2023 IEEE 31st Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)
Copyright Statement
Copyright © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Identifier
https://ieeexplore.ieee.org/abstract/document/10171508
Source
International Symposium On Field-Programmable Custom Computing Machines
Publication Status
Published
Start Date
2023-05-08
Finish Date
2023-05-11
Coverage Spatial
Marina Del Rey
Date Publish Online
2023-07-10