Magnesium hydroxide sulfate hydrate (MHSH) whiskers are used to reinforce magnesium silicate hydrate (M-S-H) cement mortars as novel microfibrous materials because of their similar pH. The microstructure, mechanical performance, and reinforcement mechanism were investigated, and the results showed that the addition of between 1 and 5 wt% MHSH whiskers improved the compressive and flexural strengths of M-S-H cement mortars. The optimal compressive and flexural strengths were obtained at MHSH whisker contents between 3 and 4 wt%. The MHSH whiskers had a limited effect on the toughness of M-S-H cement, and mortars reinforced with MHSH whiskers exhibited brittle failure due to the small size of MHSH whiskers and low fiber bridging traction. Scanning electron microscopy (SEM) revealed that the microscale reinforcement mechanism of MHSH whiskers involved whisker pullout, crack deflection, whisker-cement coalition pullout, and whisker fracture. These mechanisms helped dissipate energy and optimize the stress distribution and transfer, which were crucial to improving the flexural strength. The SEM images revealed the rough and grooved surfaces of MHSH whiskers, and X-ray photoelectron spectroscopy (XPS) showed the presence of polar functional groups on the surface which resulted in the adhesion of M-S-H gel on MHSH whiskers due to good interfacial bonding. The mercury intrusion porosimetry (MIP) results indicated that the addition of MHSH whiskers reduced the porosity of M-S-H cement mortars, which also contributed to the increased compressive strength.