One of the grand challenges in contemporary systems chemistry research is to mimic life-like functions using simple synthetic molecular networks. This is particularly true for systems that are out of chemical equilibrium and show complex dynamic behaviour, such as multi-stability, oscillations and chaos. We report here on thiodepsipeptide-based non-enzymatic networks propelled by reversible replication processes out of equilibrium, displaying bistability. Accordingly, we present quantitative analyses of the bistable behaviour, featuring a phase transition from the simple equilibration processes taking place in reversible dynamic chemistry into the bistable region. This behaviour is observed only when the system is continuously fueled by a reducing agent that keeps it far from equilibrium, and only when operating within a specifically defined parameter space. We propose that the development of biomimetic bistable systems will pave the way towards the study of more elaborate functions, such as information transfer and signalling.