Running neural networks on the NIC
File(s)
Working paper
Author(s)
Type
Working Paper
Abstract
In this paper we show that the data plane of commodity programmable (Network
Interface Cards) NICs can run neural network inference tasks required by packet
monitoring applications, with low overhead. This is particularly important as
the data transfer costs to the host system and dedicated machine learning
accelerators, e.g., GPUs, can be more expensive than the processing task
itself. We design and implement our system -- N3IC -- on two different NICs and
we show that it can greatly benefit three different network monitoring use
cases that require machine learning inference as first-class-primitive. N3IC
can perform inference for millions of network flows per second, while
forwarding traffic at 40Gb/s. Compared to an equivalent solution implemented on
a general purpose CPU, N3IC can provide 100x lower processing latency, with
1.5x increase in throughput.
Interface Cards) NICs can run neural network inference tasks required by packet
monitoring applications, with low overhead. This is particularly important as
the data transfer costs to the host system and dedicated machine learning
accelerators, e.g., GPUs, can be more expensive than the processing task
itself. We design and implement our system -- N3IC -- on two different NICs and
we show that it can greatly benefit three different network monitoring use
cases that require machine learning inference as first-class-primitive. N3IC
can perform inference for millions of network flows per second, while
forwarding traffic at 40Gb/s. Compared to an equivalent solution implemented on
a general purpose CPU, N3IC can provide 100x lower processing latency, with
1.5x increase in throughput.
Date Issued
2020-09-04
Citation
2020
Publisher
arXiv
Copyright Statement
© 2020 The Author(s)
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Identifier
http://arxiv.org/abs/2009.02353v1
Grant Number
EP/N028260/2
Subjects
cs.DC
cs.DC
cs.AI
cs.NI
Publication Status
Published