This thesis focuses on three fundamental aspects of biological systems; namely, entropy production, Bayesian mechanics, and the free-energy principle. The contributions are threefold: 1) We compute the entropy production for a greater class of systems than before, including almost any stationary diffusion process, such as degenerate diffusions where the driving noise does not act on all coordinates of the system. Importantly, this class of systems encompasses Markovian approximations of stochastic differential equations driven by colored noise, which is significant since biological systems at the macro- and meso-scale are generally subject to colored fluctuations. 2) We develop a Bayesian mechanics for biological and physical entities that interact with their environment in which we give sufficient and necessary conditions for the internal states of something to infer its external states, consistently with variational Bayesian inference in statistics and theoretical neuroscience. 3) We refine the constraints on Bayesian mechanics to obtain a description that is more specific to biological systems, called the free-energy principle. This says that active and internal states of biological systems unfold as minimising a quantity known as free energy. The mathematical foundation to the free-energy principle, presented here, unlocks a first principles approach to modeling and simulating behavior in neurobiology and artificial intelligence, by minimising free energy given a generative model of external and sensory states.