It is uncertain whether small vessel disease underlies the relationship between Type 2 Diabetes Mellitus (T2DM) and brain atrophy. We aimed to study whether retinal vascular architecture, as a proxy for cerebral small vessel disease, may modify or mediate the associations of T2DM with brain volumes. In this cross-sectional study using Magnetic Resonance Imaging (MRI) scans and retinal photographs in 451 people with and without T2DM, we measured brain volumes, geometric measures of retinal vascular architecture, clinical retinopathy, and MRI cerebrovascular lesions. There were 270 people with (mean age 67.3 years) and 181 without T2DM (mean age 72.9 years). T2DM was associated with lower gray matter volume (). T2DM was associated with greater arteriolar diameter () and optimality ratio (), but these associations were attenuated by adjustments for age and sex. Only optimality ratio was associated with lower gray matter volume (). The inclusion of retinal measures in regression models did not attenuate the association of T2DM with gray matter volume. The association of T2DM with lower gray matter volume was independent of retinal vascular architecture and clinical retinopathy. Retinal vascular measures or retinopathy may not be sufficiently sensitive to confirm a microvascular basis for T2DM-related brain atrophy.