Adaptive neural trees
File(s)
Author(s)
Tanno, Ryutaro
Arulkumaran, Kai
Alexander, Daniel C
Criminisi, Antonio
Nori, Aditya
Type
Working Paper
Abstract
Deep neural networks and decision trees operate on largely separate
paradigms; typically, the former performs representation learning with
pre-specified architectures, while the latter is characterised by learning
hierarchies over pre-specified features with data-driven architectures. We
unite the two via adaptive neural trees (ANTs) that incorporates representation
learning into edges, routing functions and leaf nodes of a decision tree, along
with a backpropagation-based training algorithm that adaptively grows the
architecture from primitive modules (e.g., convolutional layers). We
demonstrate that, whilst achieving competitive performance on classification
and regression datasets, ANTs benefit from (i) lightweight inference via
conditional computation, (ii) hierarchical separation of features useful to the
task e.g. learning meaningful class associations, such as separating natural
vs. man-made objects, and (iii) a mechanism to adapt the architecture to the
size and complexity of the training dataset.
paradigms; typically, the former performs representation learning with
pre-specified architectures, while the latter is characterised by learning
hierarchies over pre-specified features with data-driven architectures. We
unite the two via adaptive neural trees (ANTs) that incorporates representation
learning into edges, routing functions and leaf nodes of a decision tree, along
with a backpropagation-based training algorithm that adaptively grows the
architecture from primitive modules (e.g., convolutional layers). We
demonstrate that, whilst achieving competitive performance on classification
and regression datasets, ANTs benefit from (i) lightweight inference via
conditional computation, (ii) hierarchical separation of features useful to the
task e.g. learning meaningful class associations, such as separating natural
vs. man-made objects, and (iii) a mechanism to adapt the architecture to the
size and complexity of the training dataset.
Date Issued
2019-02-13
Citation
2019
Identifier
http://arxiv.org/abs/1807.06699v5
Subjects
cs.NE
cs.NE
cs.CV
cs.LG
stat.ML
Notes
International Conference on Machine Learning 2019