Dielectric nanoantennas have recently emerged as promising elements for nonlinear and ultrafast nanophotonics due to their ability to concentrate light on the nanometer scale with low losses, while exhibiting large nonlinear susceptibilities. In this work, we demonstrate that single Si nanodisks covered with a thin 30 nm thick layer of Au can generate positive and negative sub-20 fs reflectivity modulations of ∼0.3% in the vicinity of the first-order anapole mode, when excited around the second-order anapole mode. The experimental results, characterized in the visible to near-infrared spectral range, suggest that the nonlinear optical Kerr effect is the responsible mechanism for the observed all-optical switching phenomena. These findings represent an important step toward nanoscale ultrafast all-optical signal processing.