Implementing precision fertilization to maximize crop yield while minimizing economic and environmental impacts has become critical for agriculture. Variability in biomass response to fertilization within fields, among regions, and over time creates simultaneous risks of under-yielding and overfertilization. We quantify factors determining fertilization responsiveness (i.e., biomass increases with fertilization) up to 15 years in 61 unfertilized rangelands on six continents. We demonstrate widespread multi-year variability in responsiveness, with fertilization increasing average yield by 43% but failing to improve biomass 26% of the time. All sites were responsive at least once, but only four of 61 responded in all plots and years. Modelled management scenarios highlighted that fertilizer cessation is likely to generate sizable economic savings but always reduces yield because of the difficulty in predicting when and where biomass will be unresponsive. This work reveals substantial scale-dependent variability in fertilization responsiveness globally, while clarifying the prospects and pitfalls of managing more spatially and temporally precise nutrient application.