Knee implant loosening is mainly caused by the weakness of the prosthesis-bone interface and is the main reason for surgical revisions. However, pre-operative diagnosis is difficult due to lack of accurate tests. In this study, we developed a vibration-based system to detect the loosening of the tibial implant of an instrumented knee prosthesis. The proposed system includes an instrumented vibrator for transcutaneous stimulation of the bone in a repeatable manner, and accelerometer sensors integrated into the implants to measure the propagated vibration. A coherence-based detection technique was proposed to distinguish the loosened implants from the secure ones. Fourteen ex vivo lower limbs were used, on which the knee prosthesis was implanted, and harmonic-forced vibration was applied on the tibia. The input–output coherence measure provided 92.26% accuracy, a high sensitivity (91.67%) and specificity (92.86%). This technique was benchmarked against power spectrum based analysis of the propagated vibration to the implant. In particular, loosening detection based on new peak appearance, peak shift, and peak flattening in power spectra showed inferior performance to the proposed coherence-based technique. As such, application of vibration on our instrumented knee prosthesis together with input–output coherence analysis enabled us to distinguish the secure from loose implants.