Phase shifting profilometry (PSP) has been widely used in structured-light (SL) system for three-dimensional (3D) shape measurements, but the speed of PSP technique is limited by the increased phase-shifting patterns. This paper proposes an accurate and dynamic 3D shape measurement method by projecting only four patterns including three-step phase-shifting patterns and one speckle pattern. Three-step phase-shifting images are used to obtain the initial unwrapped phase map with phase ambiguity. Based on the principle of digital image correlation (DIC) and multi-view geometry, the absolute phase can be recovered reliably without requiring any embedded features or pre-defined information of the object. To improve the measurement accuracy, the projector coordinate is used as the measuring coordinate to establish a novel stereo structured-light system model. By solving a least square solution using the triple-view information, accurate 3D surface data can be reconstructed. The experimental results indicate that the proposed method can perform high-speed and accurate 3D shape measurements with an accuracy of 10.64 μm, which is superior to conventional methods and has certain instructive significance for 3D profilometry and measurement engineering.