The application of ferroelectric materials (i.e. solids that exhibit spontaneous electric polarisation) in solar cells has a long and controversial history. This includes the first observations of the anomalous photovoltaic effect (APE) and the bulk photovoltaic effect (BPE). The recent successful application of inorganic and hybrid perovskite structured materials (e.g. BiFeO3, CsSnI3, CH3NH3PbI3) in solar cells emphasises that polar semiconductors can be used in conventional photovoltaic architectures. We review developments in this field, with a particular emphasis on the materials known to display the APE/BPE (e.g. ZnS, CdTe, SbSI), and the theoretical explanation. Critical analysis is complemented with first-principles calculation of the underlying electronic structure. In addition to discussing the implications of a ferroelectric absorber layer, and the solid state theory of polarisation (Berry phase analysis), design principles and opportunities for high-efficiency ferroelectric photovoltaics are presented.