In this paper, we use probabilistic methods to determine the mean first passage time (MFPT) for a two-state piecewise deterministic Markov process (PDMP), also known as a dichotomous noise process, to escape from a finite interval. In particular, we consider the case where the set of functions generating the piecewise deterministic dynamics have one or more critical points. In order to solve this type of problem, we partition the domain into a set of subintervals that contain no critical points and impose conditions at the critical points separating these regions. Our analysis exploits the fact that a PDMP satisfies the strong Markov property. We prove that in the absence of common critical points, the MFPT is finite. Through specific examples, we also explore how the MFPT depends on the number of critical points and prove that the MFPT can be infinite if there are common critical points.