In this thesis we address the problem of data-driven coarse-graining, i.e. the process of inferring simplified models, which describe the evolution of the essential characteristics of a complex system, from available data (e.g. experimental observation or simulation data). Specifically, we consider the case where the coarse-grained model can be formulated as a stochastic differential equation. The main part of this work is concerned with data-driven coarse-graining when the underlying complex system is characterised by processes occurring across two widely separated time scales. It is known that in this setting commonly used statistical techniques fail to obtain reasonable estimators for parameters in the coarse-grained model, due to the multiscale structure of the data. To enable reliable data-driven coarse-graining techniques for diffusion processes with multiple time scales, we develop a novel estimation procedure which decisively relies on combining techniques from mathematical statistics and numerical analysis. We demonstrate, both rigorously and by means of extensive simulations, that this methodology yields accurate approximations of coarse-grained SDE models. In the final part of this work, we then discuss a systematic framework to analyse and predict complex systems using observations. Specifically, we use data-driven techniques to identify simple, yet adequate, coarse-grained models, which in turn allow to study statistical properties that cannot be investigated directly from the time series. The value of this generic framework is exemplified through two seemingly unrelated data sets of real world phenomena.