Policymakers are seeking a transformation of the energy system driven by concerns about climate change, energy security and affordability. At the same time, emerging developments in underpinning science and engineering are opening up new possibilities across the whole technology spectrum covering renewables and other supply side technologies, energy demand and energy infrastructure. This paper reviews both the “policy pull” for energy innovation activities and the “science and technology push”. It explores the expectations of a variety of organisations in both the public and private sector regarding these pressures and possibilities by assessing various scenarios and outlook exercises that have been published since 2013. It reveals a wide range of beliefs about the future development of the energy system. The paper then moves on to analyse private sector expenditure on energy research and development (R&D) and public sector budgets for energy R&D and demonstration (RD&D). This analysis demonstrates significant divergences in patterns of innovation between the private and public sectors and leads to the hypothesis that the private sector is, broadly, taking measures to reinforce the existing energy paradigm while the public sector is focusing on new energy technologies that support wider policy objectives. This pattern is consistent with past technological transitions, with innovation efforts that would transform the energy system being counteracted by countervailing efforts that reinforce the existing fossil fuel-based paradigm.