During the Salzburg Global Seminar Session 565—‘Better Health Care: How do we learn about improvement?’, participants discussed the need to unpack the ‘black box’ of improvement. The ‘black box’ refers to the fact that when quality improvement interventions are described or evaluated, there is a tendency to assume a simple, linear path between the intervention and the outcomes it yields. It is also assumed that it is enough to evaluate the results without understanding the process of by which the improvement took place. However, quality improvement interventions are complex, nonlinear and evolve in response to local settings. To accurately assess the effectiveness of quality improvement and disseminate the learning, there must be a greater understanding of the complexity of quality improvement work. To remain consistent with the language used in Salzburg, we refer to this as ‘unpacking the black box’ of improvement. To illustrate the complexity of improvement, this article introduces four quality improvement case studies. In unpacking the black box, we present and demonstrate how Cynefin framework from complexity theory can be used to categorize and evaluate quality improvement interventions. Many quality improvement projects are implemented in complex contexts, necessitating an approach defined as ‘probe-sense-respond’. In this approach, teams experiment, learn and adapt their changes to their local setting. Quality improvement professionals intuitively use the probe-sense-respond approach in their work but document and evaluate their projects using language for ‘simple’ or ‘complicated’ contexts, rather than the ‘complex’ contexts in which they work. As a result, evaluations tend to ask ‘How can we attribute outcomes to the intervention?’, rather than ‘What were the adaptations that took place?’. By unpacking the black box of improvement, improvers can more accurately document and describe their interventions, allowing evaluators to ask the right questions and more adequately evaluate quality improvement interventions.