Global sea-level (eustatic) histories generated by backstripping stratigraphy are predicated upon the lithosphere having a well understood tectonic history. However, sub-plate processes play a role in governing lithospheric vertical motions with timescales and amplitudes akin to eustasy, which are difficult to predict. We examine how stratigraphic and geophysical observations combined with simple isostatic models can be used to disentangle histories of sub-plate support and eustasy. We focus on the passive margin of Eastern North America, where a generally accepted history of eustasy has been estimated. Negative long wavelength free-air gravity anomalies, residual ocean-age depth estimates, fast upper mantle shear wave velocities, and geodynamic models suggest that Cenozoic evolution of this passive margin has been influenced by upper mantle drawdown. We build on existing analyses to backstrip sixteen wells, which, combined with seismic data, constrain timing and extent of Cenozoic subsidence. Results indicate up to ∼1000 m of water-loaded subsidence between ∼20–0 Ma centered on the Baltimore Canyon Trough. Seismic data from the trough shows Neogene aggrading clinoforms. There is little evidence for faulting or stratigraphic growth, which indicates that Neogene lithospheric strain rates were low. Amplitude and spatial extent of Neogene subsidence are difficult to explain by glacio-eustasy or glacio-isostatic adjustment. Instead, sub-plate support calculated from conversion of shear wave velocities to temperature and isostatic calculations indicate that upper mantle drawdown was responsible for subsidence of the margin. Because mantle convection is vigorous such observations are expected throughout the stratigraphic archive.