Generative optimization networks for memory efficient data generation
File(s)
Accepted version
Author(s)
Tuli, Shreshth
Tuli, Shikhar
Casale, Giuliano
Jennings, Nicholas R
Type
Conference Paper
Abstract
In standard generative deep learning models, such as autoencoders or GANs,
the size of the parameter set is proportional to the complexity of the
generated data distribution. A significant challenge is to deploy
resource-hungry deep learning models in devices with limited memory to prevent
system upgrade costs. To combat this, we propose a novel framework called
generative optimization networks (GON) that is similar to GANs, but does not
use a generator, significantly reducing its memory footprint. GONs use a single
discriminator network and run optimization in the input space to generate new
data samples, achieving an effective compromise between training time and
memory consumption. GONs are most suited for data generation problems in
limited memory settings. Here we illustrate their use for the problem of
anomaly detection in memory-constrained edge devices arising from attacks or
intrusion events. Specifically, we use a GON to calculate a
reconstruction-based anomaly score for input time-series windows. Experiments
on a Raspberry-Pi testbed with two existing and a new suite of datasets show
that our framework gives up to 32% higher detection F1 scores and 58% lower
memory consumption, with only 5% higher training overheads compared to the
state-of-the-art.
the size of the parameter set is proportional to the complexity of the
generated data distribution. A significant challenge is to deploy
resource-hungry deep learning models in devices with limited memory to prevent
system upgrade costs. To combat this, we propose a novel framework called
generative optimization networks (GON) that is similar to GANs, but does not
use a generator, significantly reducing its memory footprint. GONs use a single
discriminator network and run optimization in the input space to generate new
data samples, achieving an effective compromise between training time and
memory consumption. GONs are most suited for data generation problems in
limited memory settings. Here we illustrate their use for the problem of
anomaly detection in memory-constrained edge devices arising from attacks or
intrusion events. Specifically, we use a GON to calculate a
reconstruction-based anomaly score for input time-series windows. Experiments
on a Raspberry-Pi testbed with two existing and a new suite of datasets show
that our framework gives up to 32% higher detection F1 scores and 58% lower
memory consumption, with only 5% higher training overheads compared to the
state-of-the-art.
Date Acceptance
2021-12-01
Copyright Statement
© 2021 The Author(s).
Identifier
http://arxiv.org/abs/2110.02912v4
Source
Workshop on ML for Systems at NeurIPS
Subjects
cs.LG
cs.LG
Notes
Accepted in NeurIPS 2021 - Workshop on ML for Systems
Publication Status
Accepted
Start Date
2021-12-13
Finish Date
2021-12-13
Coverage Spatial
Virtual event