Passivity is a fundamental concept in thermodynamics that demands a quantum system’s energy cannot be lowered by any reversible, unitary process acting on the system. In the limit of many such systems, passivity leads in turn to the concept of complete passivity, thermal states and the emergence of a thermodynamic temperature. Here we only consider a single system and show that every passive state except the thermal state is unstable under a weaker form of reversibility. Indeed, we show that given a single copy of any athermal quantum state, an optimal amount of energy can be extracted from it when we utilise a machine that operates in a reversible cycle. This means that for individual systems, the only form of passivity that is stable under general reversible processes is complete passivity, and thus provides a physically motivated identification of thermal states when we are not operating in the thermodynamic limit.