Compared to fullerene based electron acceptors, n-type organic semiconductors, so-called non-fullerene acceptors (NFAs), possess some distinct advantages, such as readily tuning of optical absorption and electronic energy levels, strong absorption in the visible region and good morphological stability for flexible electronic devices. The design and synthesis of new NFAs have enabled the power conversion efficiencies (PCEs) of organic photovoltaic (OPV) devices to increase to around 19%. This review summarises the important breakthroughs that have contributed to this progress, focusing on three classes of NFAs, i.e. perylene diimide (PDI), diketopyrrolopyrrole (DPP) and acceptor–donor–acceptor (A-D-A) based NFAs. Specifically, the PCEs of PDI, DPP, and A-D-A series based non-fullerene OPVs have been reported up to 11%, 13% and 19%, respectively. Structure–property relationships of representative NFAs and their impact on OPV performances are discussed. Finally, we consider the remaining challenges and promising directions for achieving high-performing NFAs.