Label free tracking of small bio-particles such as proteins or viruses is of great utility in the study of biological processes, however such experiments are frequently hindered by weak signal strengths and a susceptibility to scattering impurities. To overcome these problems we here propose a novel technique leveraging the enhanced sensitivity of both interferometric detection and the strong field confinement of surface plasmons. Specifically, we show that interference between the field scattered by an analyte particle and a speckle reference field, derived from random scattering of surface plasmons propagating on a rough metal film, enables particle tracking with sub-wavelength accuracy. We present the analytic framework of our technique and verify its robustness to noise through Monte Carlo simulations.