Stochastic Processes are ubiquitous, from automated engineering, through financial
markets, to space exploration. These systems are typically highly dynamic, unpredictable
and resistant to analytic methods; coupled with a need to orchestrate long control
sequences which are both highly complex and uncertain. This report examines some existing
single- and multi-agent modelling frameworks, details their strengths and weaknesses,
and uses the experience to identify some fundamental tenets of good practice in modelling
stochastic processes. It goes on to develop a new family of frameworks based on these tenets,
which can model single- and multi-agent domains with equal clarity and flexibility, while
remaining close enough to the existing frameworks that existing analytic and learning tools
can be applied with little or no adaption. Some simple and larger examples illustrate the
similarities and differences of this approach, and a discussion of the challenges inherent in
developing more flexible tools to exploit these new frameworks concludes matters.