A robust immersed boundary method for semi-implicit discretizations of the Navier-Stokes equations on curvilinear grids is presented. No-slip conditions are enforced via momentum forcing, and mass conservation at the immersed boundary is satisfied via a mass source term developed for moving bodies. The errors associated with an explicit evaluation of the momentum forcing are analysed, and their influence on the stability of the underlying Navier-Stokes solver is examined. An iterative approach to compute the forcing term implicitly is proposed, which reduces the errors at the boundary and retains the stability guarantees of the original semi-implicit discretization of the Navier-Stokes equations. The implementation in generalized curvilinear coordinates and the treatment of moving boundaries are presented, followed by a number of test cases. The tests include stationary and moving boundaries and curvilinear grid problems (decaying vortex problem, stationary cylinder, flow in 90° bend in circular duct and oscillating cylinder in fluid at rest).