It has been postulated that autism spectrum disorder is underpinned by an ‘atypical connectivity’ involving higher-order association brain regions. To test this hypothesis in a large cohort of adults with autism spectrum disorder we compared the white matter networks of 61 adult males with autism spectrum disorder and 61 neurotypical controls, using two complementary approaches to diffusion tensor magnetic resonance imaging. First, we applied tract-based spatial statistics, a ‘whole brain’ non-hypothesis driven method, to identify differences in white matter networks in adults with autism spectrum disorder. Following this we used a tract-specific analysis, based on tractography, to carry out a more detailed analysis of individual tracts identified by tract-based spatial statistics. Finally, within the autism spectrum disorder group, we studied the relationship between diffusion measures and autistic symptom severity. Tract-based spatial statistics revealed that autism spectrum disorder was associated with significantly reduced fractional anisotropy in regions that included frontal lobe pathways. Tractography analysis of these specific pathways showed increased mean and perpendicular diffusivity, and reduced number of streamlines in the anterior and long segments of the arcuate fasciculus, cingulum and uncinate—predominantly in the left hemisphere. Abnormalities were also evident in the anterior portions of the corpus callosum connecting left and right frontal lobes. The degree of microstructural alteration of the arcuate and uncinate fasciculi was associated with severity of symptoms in language and social reciprocity in childhood. Our results indicated that autism spectrum disorder is a developmental condition associated with abnormal connectivity of the frontal lobes. Furthermore our findings showed that male adults with autism spectrum disorder have regional differences in brain anatomy, which correlate with specific aspects of autistic symptoms. Overall these results suggest that autism spectrum disorder is a condition linked to aberrant developmental trajectories of the frontal networks that persist in adult life.