High–refractive index nanostructured dielectrics have the ability to locally enhance electromagnetic fields with low losses while presenting high third-order nonlinearities. In this work, we exploit these characteristics to achieve efficient ultrafast all-optical modulation in a crystalline gallium phosphide (GaP) nanoantenna through the optical Kerr effect (OKE) and two-photon absorption (TPA) in the visible/near-infrared range. We show that an individual GaP nanodisk can yield differential reflectivity modulations of up to ~40%, with characteristic modulation times between 14 and 66 fs, when probed at the anapole excitation (AE). Numerical simulations reveal that the AE represents a unique condition where both the OKE and TPA contribute with the same modulation sign, maximizing the response. These findings highly outperform previous reports on sub–100-fs all-optical switching from resonant nanoscale dielectrics, which have demonstrated modulation depths no larger than 0.5%, placing GaP nanoantennas as a promising choice for ultrafast all-optical modulation at the nanometer scale.