We demonstrate that light spectator fields in their equilibrium can source sizeable CMB anisotropies through modulated reheating even in the absence of direct couplings to the inflaton. The effect arises when the phase space of the inflaton decay is modulated by the spectator which generates masses for the decay products. We call the mechanism indirect modulation and using the stochastic eigenvalue expansion show that it can source perturbations even four orders of magnitude larger than the observed amplitude. Importantly, the indirect mechanism is present in the Standard Model extended with right- handed neutrinos. For a minimally coupled Higgs boson this leads to a novel lower bound on the quartic coupling and constrains the neutrino Yukawas below unity.