This work presents an approach for exploring unknown surfaces with discontinuities using only force/torque information. The motivation is to build an information map of an unknown object or environment by performing a fully-autonomous haptic exploration. Examples of discontinuities considered here are contours with sharp turns (such as wall corners) and abrupt dips (such as cliffs). Compliant motion control using force information has the ability to conform to unknown, smooth surfaces but not to discontinuous surfaces. This paper investigates solutions to address the limitation in compliant motion control over discontinuities while maintaining a desired normal force along the surface. We propose two methods to address the problem: (1) superposition of motion and force control and (2) rotation of axes for force and motion control. The theoretical principles are discussed and experimental results with a KUKA lightweight arm moving in 2D space are presented. Both approaches successfully negotiate objects with sharp 90-degree and 120-degree turns while still maintaining good tracking of the desired force.