PolyLUT: ultra-low latency polynomial inference with hardware-aware structured pruning
File(s)
Preprint
OA Location
Author(s)
Andronic, Marta
Li, Jiawen
Constantinides, George A
Type
preprint
Abstract
Standard deep neural network inference involves the computation of interleaved linear maps and nonlinear activation functions. Prior work for ultra-low latency implementations has hardcoded these operations inside FPGA lookup tables (LUTs). However, FPGA LUTs can implement a much greater variety of functions. In this paper, we propose a novel approach to training DNNs for FPGA deployment using multivariate polynomials as the basic building block. Our method takes advantage of the flexibility offered by the soft logic, hiding the polynomial evaluation inside the LUTs with minimal overhead. By using polynomial building blocks, we achieve the same accuracy using considerably fewer layers of soft logic than by using linear functions, leading to significant latency and area improvements. LUTbased implementations also face a significant challenge: the LUT size grows exponentially with the number of inputs. Prior work relies on a priori fixed sparsity, with results heavily dependent on seed selection. To address this, we propose a structured pruning strategy using a bespoke hardware-aware group regularizer that encourages a particular sparsity pattern that leads to a small number of inputs per neuron. We demonstrate the effectiveness of PolyLUT on three tasks: network intrusion detection, jet identification at the CERN Large Hadron Collider, and MNIST.
Date Issued
2025-01-14
Citation
arXiv, 2025
Journal / Book Title
arXiv
Copyright Statement
Copyright © 2025 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/).
License URL
Identifier
http://arxiv.org/abs/2501.08043v1
Subjects
Science & Technology
Technology
Computer Science, Hardware & Architecture
Engineering, Electrical & Electronic
Computer Science
Engineering
Table lookup
Training
Neurons
Biological neural networks
Accuracy
Polynomials
Logic
Vectors
Real-time systems
LUT-based neural networks
soft logic
field-programmable gate array
deep learning inference
hardware accelerator
NETWORKS