Infants born in early term (37-38 weeks gestation) experience slower neurodevelopment than those born at full term (40-41 weeks gestation). While this could be due to higher perinatal morbidity, gestational age at birth may also have a direct effect on the brain. Here we characterise brain volume and white matter correlates of gestational age at birth in healthy term-born neonates and their relationship to later neurodevelopmental outcome using T2 and diffusion weighted MRI acquired in the neonatal period from a cohort (n = 454) of healthy babies born at term age (>37 weeks gestation) and scanned between 1 and 41 days after birth. Images were analysed using tensor-based morphometry and tract-based spatial statistics. Neurodevelopment was assessed at age 18 months using the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III). Infants born earlier had higher relative ventricular volume and lower relative brain volume in the deep grey matter, cerebellum and brainstem. Earlier birth was also associated with lower fractional anisotropy, higher mean, axial, and radial diffusivity in major white matter tracts. Gestational age at birth was positively associated with all Bayley-III subscales at age 18 months. Regression models predicting outcome from gestational age at birth were significantly improved after adding neuroimaging features associated with gestational age at birth. This work adds to the body of evidence of the impact of early term birth and highlights the importance of considering the effect of gestational age at birth in future neuroimaging studies including term-born babies.