Permuting accumulation order for low-precision machine learning
File(s)
Accepted version
Author(s)
Samson, Ebby
Liu, Tony
Luk, Wayne
Constantinides, George A
Type
Conference Paper
Abstract
Quantization of weights and activations in neural networks is widely used to reduce data movement and the computational footprint of multipliers in arithmetic units. However, this increases the relative area contribution of adders. Most recent work in neural network quantization uses large floating-point accumulators due the large rounding and clipping errors incurred by smaller accumulators even if weights and activations are otherwise quantized to narrow data types. In this work, we propose a novel method of finding and applying permutations to weight and activation order in neural networks to reduce the error induced by small floating-point adders for the multiply accumulate (MAC) functions in matrix multiplications. Our method optimizes the order of accumulation with very low computational overhead by using a shared ideal order for a group of vectors instead of an ideal order for each vector, and using a static order rather than dynamically generating one at runtime. Our technique does not require quantization-aware training (QAT) or modification of weights, making it applicable to large language models (LLMs).
Date Issued
2025-08-20
Date Acceptance
2025-07-01
Citation
2025 IEEE 36th International Conference on Application-specific Systems, Architectures and Processors (ASAP), 2025, pp.176-177
ISSN
2160-0511
Publisher
IEEE
Start Page
176
End Page
177
Journal / Book Title
2025 IEEE 36th International Conference on Application-specific Systems, Architectures and Processors (ASAP)
Copyright Statement
Copyright © 2025, IEEE. This is the author’s accepted manuscript made available under a CC-BY licence in accordance with Imperial’s Research Publications Open Access policy (www.imperial.ac.uk/oa-policy)
License URL
Source
2025 IEEE 36th International Conference on Application-specific Systems, Architectures and Processors (ASAP)
Publication Status
Published
Start Date
2025-07-28
Finish Date
2025-07-30
Coverage Spatial
Vancouver, BC, Canada