Feature selection has been studied in machine
learning and data mining for many years, and
is a valuable way to improve classification accu-
racy while reducing model complexity. Two main
classes of feature selection methods - filter and
wrapper - discard those features which are not se-
lected, and do not consider them in the predictive
model. We propose that these unselected features
may instead be used as an additional source of in-
formation at train time. We describe a strategy
called Learning using Unselected Features (LUFe)
that allows selected and unselected features to serve
different functions in classification. In this frame-
work, selected features are used directly to set
the decision boundary, and unselected features are
utilised in a secondary role, with no additional cost
at test time. Our empirical results on 49 textual
datasets show that LUFe can improve classification
performance in comparison with standard wrapper
and filter feature selection.