For nearly any challenging scientific problem
evaluation of the likelihood is problematic if not impossible.
Approximate Bayesian computation (ABC) allows
us to employ the whole Bayesian formalism to problems
where we can use simulations from a model, but cannot
evaluate the likelihood directly. When summary statistics of
real and simulated data are compared—rather than the data
directly—information is lost, unless the summary statistics
are sufficient. Sufficient statistics are, however, not common
but without them statistical inference in ABC inferences
are to be considered with caution. Previously other authors
have attempted to combine different statistics in order to
construct (approximately) sufficient statistics using search
and information heuristics. Here we employ an informationtheoretical
framework that can be used to construct appropriate
(approximately sufficient) statistics by combining different
statistics until the loss of information is minimized.
We start from a potentially large number of different statistics
and choose the smallest set that captures (nearly) the
same information as the complete set. We then demonstrate
that such sets of statistics can be constructed for both parameter
estimation and model selection problems, and we apply
our approach to a range of illustrative and real-world model
selection problems.