Although studies have examined the association of the Relative Fat Mass (RFM, a novel anthropometric index used as a surrogate for whole-body fat percentage) with all-cause mortality, the association of RFM with diabetes-related mortality and heart disease mortality has not been thoroughly investigated. In addition, no study has compared the associations of RFM and waist circumference (a surrogate for intra-abdominal fat) with cause-specific mortality and all-cause mortality. In the present study, we addressed these knowledge gaps. We used data from the US National Health and Nutrition Examination Survey (NHANES) 1999–2018. NHANES III was used for validation. Analyses included 46,535 adults (mean age 46.5 years). During a median follow-up time of 9.7 years, 6,101 participants died (743 from diabetes; 1,514 from heart disease). Compared with BMI and WC, RFM was more strongly associated with diabetes-related mortality in both women and men, adjusting for age, ethnicity, education, and smoking status. All anthropometric measures were similarly strongly associated with heart disease mortality and all-cause mortality. RFM showed greater predictive discrimination of mortality. Similar results were found in NHANES III (n = 14,448). In conclusion, RFM is strongly associated with diabetes-related mortality, heart disease mortality, and all-cause mortality, and outperforms conventional adiposity measures for prediction of mortality.