This paper introduces Classification with Margin Constraints (CMC), a simple
generalization of cost-sensitive classification that unifies several learning settings.
In particular, we show that a CMC classifier can be used, out of the box, to solve
regression, quantile estimation, and several anomaly detection formulations. On
the one hand, our reductions to CMC are at the loss level: the optimization problem
to solve under the equivalent CMC setting is exactly the same as the optimization
problem under the original (e.g. regression) setting. On the other hand,
due to the close relationship between CMC and standard binary classification, the
ideas proposed for efficient optimization in binary classification naturally extend
to CMC. As such, any improvement in CMC optimization immediately transfers
to the domains reduced to CMC, without the need for new derivations or programs.
To our knowledge, this unified view has been overlooked by the existing
practice in the literature, where an optimization technique (such as SMO or PEGASOS)
is first developed for binary classification and then extended to other
problem domains on a case-by-case basis. We demonstrate the flexibility of CMC
by reducing two recent anomaly detection and quantile learning methods to CMC.