Atmospheric mass-loss is known to play a leading role in sculpting the demographics of small, close-in exoplanets. Knowledge of how such planets evolve allows one to ‘rewind the clock’ to infer the conditions in which they formed. Here, we explore the relationship between a planet’s core mass and its atmospheric mass after protoplanetary disc dispersal by exploiting XUV photoevaporation as an evolutionary process. Historically, this inference problem would be computationally infeasible due to the large number of planet models required; however, we use a novel atmospheric evolution emulator which utilizes neural networks to provide three orders of magnitude in speedup. First, we provide a proof of concept for this emulator on a real problem by inferring the initial atmospheric conditions of the TOI-270 multi-planet system. Using the emulator, we find near-indistinguishable results when compared to the original model. We then apply the emulator to the more complex inference problem, which aims to find the initial conditions for a sample of Kepler, K2, and TESS planets with well-constrained masses and radii. We demonstrate that there is a relationship between core masses and the atmospheric mass they retain after disc dispersal. This trend is consistent with the ‘boil-off’ scenario, in which close-in planets undergo dramatic atmospheric escape during disc dispersal. Thus, it appears that the exoplanet population is consistent with the idea that close-in exoplanets initially acquired large massive atmospheres, the majority of which is lost during disc dispersal, before the final population is sculpted by atmospheric loss over 100 Myr to Gyr time-scales.