We explore probability modelling of discretization uncertainty for system
states defined implicitly by ordinary or partial differential equations. Accounting
for this uncertainty can avoid posterior under-coverage when likelihoods are
constructed from a coarsely discretized approximation to system equations. A formalism
is proposed for inferring a fixed but a priori unknown model trajectory
through Bayesian updating of a prior process conditional on model information.
A one-step-ahead sampling scheme for interrogating the model is described, its
consistency and first order convergence properties are proved, and its computational
complexity is shown to be proportional to that of numerical explicit one-step
solvers. Examples illustrate the flexibility of this framework to deal with a wide
variety of complex and large-scale systems. Within the calibration problem, discretization
uncertainty defines a layer in the Bayesian hierarchy, and a Markov
chain Monte Carlo algorithm that targets this posterior distribution is presented.
This formalism is used for inference on the JAK-STAT delay differential equation
model of protein dynamics from indirectly observed measurements. The discussion
outlines implications for the new field of probabilistic numerics.