Piecewise-Deterministic Markov Processes (PDPs) are a general class of non- diffusion stochastic system models. A PDP is a Markov process that follows deterministic trajectories between random jumps, the latter occurs either spontaneously, in a Poisson-like fashion, or when the process hits the boundary of its state space. This formulation includes a wide variety of applied problems in engineering, operations research, management science and economics; examples include queueing systems, optimal exploration and consumption of renewable or non-renewable resources, capacity expansion, Permanent Health Insurance (PHI), insurance analysis, target tracking and fault detection in process systems. As a preliminary example of piecewise-deterministic processes, an iterative computational method for determining the value function of an optimal control problem, related to target tracking, was obtained. The target is assumed to be located in a fixed known position in space but its identity (hostile or friendly) is only known with a prior probability. An observation of the target can be made at any location and its error has position-dependent probability. The objective was finding the optimal navigation and observation strategy which leads to a final decision (i.e. the target is friendly or hostile). The value function was shown to be the unique viscosity solution of some variational inequality and in addition the unique fixed point of a non-decreasing concave operator. The strong boundary condition in the theory of PDPs (introduced by Vermes [Ver85]) which states that all trajectories in the vicinity of the boundary must steer towards the boundary, was relaxed and under some weaker assumptions it was proved that the associated value function is the unique viscosity solution of some Dirichlet type HJB equation. The stochastic optimal control problem reduces to a deterministic exit-time optimal control problem with an unusual boundary condition and an iterative computational method can be used for finding the value function and optimal control; in fact the value function is the unique fixed point of some two-stage contraction operator and hence the convergence of the algorithm is guaranteed. As a final example, the optimal consumption and exploration of non-renewable resources, e.g. oil, is considered. The objective is to find the optimal consumption and exploration rates under uncertain exploration. The policy maker determines the consumption and exploration rates both measured in Pounds per unit time. It is shown that the exploration policy is of “bang-bang” type, i.e. either explore at the maximum rate or not at all. Some simulation results obtained via the iterative computational method are presented.