Internet-of-Things (IoT) devices are known to be the source of many security
problems, and as such, they would greatly benefit from automated management.
This requires robustly identifying devices so that appropriate network security
policies can be applied. We address this challenge by exploring how to
accurately identify IoT devices based on their network behavior, while
leveraging approaches previously proposed by other researchers.
We compare the accuracy of four different previously proposed machine
learning models (tree-based and neural network-based) for identifying IoT
devices. We use packet trace data collected over a period of six months from a
large IoT test-bed. We show that, while all models achieve high accuracy when
evaluated on the same dataset as they were trained on, their accuracy degrades
over time, when evaluated on data collected outside the training set. We show
that on average the models' accuracy degrades after a couple of weeks by up to
40 percentage points (on average between 12 and 21 percentage points). We argue
that, in order to keep the models' accuracy at a high level, these need to be
continuously updated.