Inspired by the considerable success of cryogenically cooled NMR cryoprobes, we present an upgraded X-band EPR probehead, equipped with a cryogenic low-noise preamplifier. Our setup suppresses source noise, can handle the high microwave powers typical in X-band pulsed EPR, and is compatible with the convenient resonator coupling and sample access found on commercially available spectrometers. Our approach allows standard pulsed and continuous-wave EPR experiments to be performed at X-band frequency with significantly increased sensitivity compared to the unmodified setup. The probehead demonstrates a voltage signal-to-noise ratio (SNR) enhancement by a factor close to 8× at a temperature of 6 K, and remains close to 2× at room temperature. By further suppressing room-temperature noise at the expense of reduced microwave power (and thus minimum -pulse length), the factor of SNR improvement approaches 15 at 6 K, corresponding to an impressive 200-fold reduction in EPR measurement time. We reveal the full potential of this probehead by demonstrating such SNR improvements using a suite of typical hyperfine and dipolar spectroscopy experiments on exemplary samples.