As AI models become ever more complex and intertwined in humans’ daily lives, greater levels of interactivity of explainable AI (XAI) methods are needed. In this paper, we propose the use of belief change theory as a formal foundation for operators that model the incorporation of new information in machines in interactive XAI. We argue that this type of formalisation provides a framework and a methodology to develop interactive explanations in a principled manner, providing warranted behaviour and favouring transparency and accountability of such interactions. Concretely, we first define a novel, logic-based formalism to provide a representation of how explanatory information is shared between humans and machines. We then consider real world scenarios for interactive XAI, with different prioritisations of new and existing knowledge, in which our formalism may be instantiated. Finally, we analyse a core set of belief change postulates, discussing their suitability for our real world settings and pointing to particular challenges that may require the relaxation or
re-interpretation of some of the theoretical assumptions underlying existing belief change operators.