We develop an elastodynamic theory to predict the diffuse scattered field of elastic waves by randomly rough surfaces, for the first time, with the aid of the Kirchhoff approximation (KA). Analytical expressions are derived incorporating surface statistics, to represent the expectation of the angular distribution of the diffuse intensity for different modes. The analytical solutions are successfully verified with numerical Monte Carlo simulations, and also validated by comparison with experiments. We then apply the theory to quantitatively investigate the effects of the roughness and the shear-to-compressional wave speed ratio on the mode conversion and the scattering intensity, from low to high roughness within the valid region of KA. Both the direct and the mode converted intensities are significantly affected by the roughness, which leads to distinct scattering patterns for different wave modes. The mode conversion effect is very strong around the specular angle and it is found to increase as the surface appears to be more rough. In addition, the 3D roughness induced coupling between the out-of-plane shear horizontal (SH) mode and the in-plane modes is studied. The intensity of the SH mode is shown to be very sensitive to the out-of-plane correlation length, being influenced more by this than by the RMS value of the roughness. However, it is found that the depolarization pattern for the diffuse field is independent of the actual value of the roughness.