In this paper we study, for the first time explicitly, the implications of endowing an interested party (i.e. a teacher) with the ability to modify the underlying dynamics of the environment, in order to encourage an agent to learn to follow a specific policy. We introduce a cost function which can be used by the teacher to balance the modifications it makes to the underlying environment dynamics, with the learner’s performance compared to some ideal, desired, policy. We formulate teacher’s problem of determining optimal environment changes as a planning and control problem, and empirically validate the effectiveness of our model.