Hematite (α-Fe2O3) is the most studied artificial oxygen-evolving photo-anode and yet its efficiency limitations and their origin remain unknown. A sub-picosecond reorganisation of the hematite structure has been proposed as the mechanism which dictates carrier lifetimes, energetics and the ultimate conversion yields. However, the importance of this reorganisation for actual device performance is unclear. Here we report an in situ observation of charge carrier self-localisation in a hematite device, and demonstrate that this process affects recombination losses in photoelectrochemical cells. We apply an ultrafast, device-based optical-control method to resolve the subpicosecond formation of small polarons and estimate their reorganisation energy to be ~0.5 eV. Coherent oscillations in the photocurrent signals indicate that polaron formation may be coupled to specific phonon modes (<100 cm-1). Our results bring together spectroscopic and device characterisation approaches to reveal new photophysics of broadly-studied hematite devices.