Oceanic residual depth varies on urn:x-wiley:jgrb:media:jgrb54314:jgrb54314-math-0001 5,000 km wavelengths with amplitudes of ±1 km. A component of this short‐wavelength signal is dynamic topography caused by convective flow in the upper ∼300 km of the mantle. It exerts a significant influence on landscape evolution and sea level change, but its contribution is often excluded in geodynamic models of whole‐mantle flow. Using seismic tomography to resolve buoyancy anomalies in the oceanic upper mantle is complicated by the dominant influence of lithospheric cooling on velocity structure. Here, we remove this cooling signal from global surface wave tomographic models, revealing a correlation between positive residual depth and slow residual velocity anomalies at depths <300  km. To investigate whether these anomalies are of sufficient amplitude to account for short‐wavelength residual depth variations, we calibrate an experimentally derived parameterization of anelastic deformation at seismic frequencies to convert shear wave velocity into temperature, density, and diffusion creep viscosity. Asthenospheric temperature anomalies reach +150°C in the vicinity of major magmatic hot spots and correlate with geochemical and geophysical proxies for potential temperature along mid‐ocean ridges. Locally, we find evidence for a ∼150 km‐thick, low‐viscosity asthenospheric channel. Incorporating our revised density structure into models of whole‐mantle flow yields reasonable agreement with residual depth observations and suggests that ±30 km deviations in local lithospheric thickness account for a quarter of total amplitudes. These predictions remain compatible with geoid constraints and substantially improve the fit between power spectra of observed and predicted dynamic topography. This improvement should enable more accurate reconstruction of the spatiotemporal evolution of Cenozoic dynamic topography.