To scale Gaussian processes (GPs) to large data
sets we introduce the robust Bayesian Committee
Machine (rBCM), a practical and scalable
product-of-experts model for large-scale
distributed GP regression. Unlike state-of-theart
sparse GP approximations, the rBCM is conceptually
simple and does not rely on inducing
or variational parameters. The key idea is to
recursively distribute computations to independent
computational units and, subsequently, recombine
them to form an overall result. Efficient
closed-form inference allows for straightforward
parallelisation and distributed computations with
a small memory footprint. The rBCM is independent
of the computational graph and can
be used on heterogeneous computing infrastructures,
ranging from laptops to clusters. With sufficient
computing resources our distributed GP
model can handle arbitrarily large data sets.