Heat pumps (HPs) have emerged as a key technology for reducing energy use and greenhouse gas emissions. This study evaluates the potential switch to air-to-air HPs (AAHPs) in Toulouse, France, where conventional space heating is split between electric and gas sources. In this context, we find that AAHPs reduce heating energy consumption by 57% to 76%, with electric heating energy consumption decreasing by 6% to 47%, resulting in virtually no local heating-related CO2 emissions. We observe a slight reduction in near-surface air temperature of up to 0.5 °C during cold spells, attributable to a reduction in sensible heat flux, which is unlikely to compromise AAHPs operational efficiency. While Toulouse's heating energy mix facilitates large energy savings, electric energy consumption may increase in cities where gas or other fossil fuel sources prevail. Furthermore, as AAHPs efficiency varies with internal and external conditions, their impact on the electrical grid is more complex than conventional heating systems. The results underscore the importance of matching heating system transitions with sustainable electricity generation to maximize environmental benefits. The study highlights the intricate balance between technological advancements in heating and their broader environmental and policy implications, offering key insights for urban energy policy and sustainability efforts.