A multi-view image sequence provides a much richer
capacity for object recognition than from a single image.
However, most existing solutions to multi-view recognition
typically adopt hand-crafted, model-based geometric methods,
which do not readily embrace recent trends in deep
learning. We propose to bring Convolutional Neural Networks
to generic multi-view recognition, by decomposing
an image sequence into a set of image pairs, classifying
each pair independently, and then learning an object classi-
fier by weighting the contribution of each pair. This allows
for recognition over arbitrary camera trajectories, without
requiring explicit training over the potentially infinite number
of camera paths and lengths. Building these pairwise
relationships then naturally extends to the next-best-view
problem in an active recognition framework. To achieve
this, we train a second Convolutional Neural Network to
map directly from an observed image to next viewpoint.
Finally, we incorporate this into a trajectory optimisation
task, whereby the best recognition confidence is sought for
a given trajectory length. We present state-of-the-art results
in both guided and unguided multi-view recognition on the
ModelNet dataset, and show how our method can be used
with depth images, greyscale images, or both.