This paper analyses the emissions and cost impacts of mitigation of non-CO2 greenhouse gases (GHGs) at a global level, in scenarios aimed at meeting a range of long-term temperature goals (LTTGs). The study combines an integrated assessment model (TIAM-Grantham) representing CO2 emissions (and their mitigation) from the fossil fuel combustion and industrial sectors, coupled with a model covering non-CO2 emissions (GAINS), using the latest global warming potentials from the Intergovernmental Panel on Climate Change’s Fifth Assessment Report. We illustrate that in general non-CO2 mitigation measures are less costly than CO2 mitigation measures, with the majority of their abatement potential achievable at US2005$100/tCO2e or less throughout the 21st century (compared to a marginal CO2 mitigation cost which is already greater than this by 2030 in the most stringent mitigation scenario). As a result, the total cumulative discounted cost over the period 2010–2100 (at a 5% discount rate) of limiting global average temperature change to 2.5 °C by 2100 is $48 trillion (about 1.6% of cumulative discounted GDP over the period 2010–2100) if only CO2 from the fossil fuel and industrial sectors is targeted, whereas the cost falls to $17 trillion (0.6% of GDP) by including non-CO2 GHG mitigation in the portfolio of options—a cost reduction of about 65%. The criticality of non-CO2 mitigation recommends further research, given its relatively less well-explored nature when compared to CO2 mitigation.