There has been much research on shrinkage methods for real-valued covariance matrices and their inverses (precision matrices). In spectral analysis of p-vector-valued time series, complex-valued spectral matrices and precision matrices arise, and good shrinkage methods are often required, most notably when the estimated complex-valued spectral matrix is singular. As an improvement on the Ledoit-Wolf (LW) type of spectral matrix estimator we use random matrix theory to derive a Rao-Blackwell estimator for a spectral matrix, its inverse being a Rao-Blackwellized estimator for the spectral precision matrix. A random matrix method has previously been proposed for complex-valued precision matrices. It was implemented by very costly simulations. We formulate a fast, completely analytic approach. Moreover, we derive a way of selecting an important parameter using predictive risk methodology. We show that both the Rao-Blackwell estimator and the random matrix estimator of the precision matrix can substantially outperform the inverse of the LW estimator in a time series setting. Our new methodology is applied to EEG-derived time series data where it is seen to work well and deliver substantial improvements for precision matrix estimation.