Detecting low dose rates of X-rays is critical for making safer radiology instruments, but is limited by the absorber materials available. Here, we develop bismuth oxyiodide (BiOI) single crystals into effective X-ray detectors. BiOI features complex lattice dynamics, owing to the ionic character of the lattice and weak van der Waals interactions between layers. Through use of ultrafast spectroscopy, first-principles computations and detailed optical and structural characterisation, we show that photoexcited charge-carriers in BiOI couple to intralayer breathing phonon modes, forming large polarons, thus enabling longer drift lengths for the photoexcited carriers than would be expected if self-trapping occurred. This, combined with the low and stable dark currents and high linear X-ray attenuation coefficients, leads to strong detector performance. High sensitivities reaching 1.1  × 103 μC Gyair−1 cm−2 are achieved, and the lowest dose rate directly measured by the detectors was 22 nGyair s−1. The photophysical principles discussed herein offer new design avenues for novel materials with heavy elements and low-dimensional electronic structures for (opto)electronic applications.