Nonideality-aware training for accurate and robust low-power memristive neural networks
File(s)
Author(s)
Type
Journal Article
Abstract
Recent years have seen a rapid rise of artificial neural networks being
employed in a number of cognitive tasks. The ever-increasing computing
requirements of these structures have contributed to a desire for novel
technologies and paradigms, including memristor-based hardware accelerators.
Solutions based on memristive crossbars and analog data processing promise to
improve the overall energy efficiency. However, memristor nonidealities can
lead to the degradation of neural network accuracy, while the attempts to
mitigate these negative effects often introduce design trade-offs, such as
those between power and reliability. In this work, we design nonideality-aware
training of memristor-based neural networks capable of dealing with the most
common device nonidealities. We demonstrate the feasibility of using
high-resistance devices that exhibit high $I$-$V$ nonlinearity -- by analyzing
experimental data and employing nonideality-aware training, we estimate that
the energy efficiency of memristive vector-matrix multipliers is improved by
three orders of magnitude ($0.715\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$ to $381\
\mathrm{TOPs}^{-1}\mathrm{W}^{-1}$) while maintaining similar accuracy. We show
that associating the parameters of neural networks with individual memristors
allows to bias these devices towards less conductive states through
regularization of the corresponding optimization problem, while modifying the
validation procedure leads to more reliable estimates of performance. We
demonstrate the universality and robustness of our approach when dealing with a
wide range of nonidealities.
employed in a number of cognitive tasks. The ever-increasing computing
requirements of these structures have contributed to a desire for novel
technologies and paradigms, including memristor-based hardware accelerators.
Solutions based on memristive crossbars and analog data processing promise to
improve the overall energy efficiency. However, memristor nonidealities can
lead to the degradation of neural network accuracy, while the attempts to
mitigate these negative effects often introduce design trade-offs, such as
those between power and reliability. In this work, we design nonideality-aware
training of memristor-based neural networks capable of dealing with the most
common device nonidealities. We demonstrate the feasibility of using
high-resistance devices that exhibit high $I$-$V$ nonlinearity -- by analyzing
experimental data and employing nonideality-aware training, we estimate that
the energy efficiency of memristive vector-matrix multipliers is improved by
three orders of magnitude ($0.715\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$ to $381\
\mathrm{TOPs}^{-1}\mathrm{W}^{-1}$) while maintaining similar accuracy. We show
that associating the parameters of neural networks with individual memristors
allows to bias these devices towards less conductive states through
regularization of the corresponding optimization problem, while modifying the
validation procedure leads to more reliable estimates of performance. We
demonstrate the universality and robustness of our approach when dealing with a
wide range of nonidealities.
Date Issued
2022-05-04
Date Acceptance
2022-03-28
Citation
Advanced Science, 2022, 9 (17)
ISSN
2198-3844
Publisher
Wiley Open Access
Journal / Book Title
Advanced Science
Volume
9
Issue
17
Copyright Statement
© 2022 The Authors. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
License URL
Identifier
http://arxiv.org/abs/2112.06887v3
Subjects
cs.ET
cs.ET
Notes
29 pages, 20 figures, 4 tables
Publication Status
Published online
Date Publish Online
2022-05-04