Deep neural networks (NNs) are usually trained with unconstrained optimisation algorithms. With a reasoning similar to the constrained Kalman filter, incorporating known information in the form of equality constraints at certain checkpoints can potentially improve prediction accuracy. For continuous-time dynamical systems, the state constraints should be enforced in an ordinary differential equation (ODE) model which embeds NNs to represent a learned part of dynamics or a control policy. To this end, incremental correction methods are developed for post-processing of the dynamical systems modelled with NNs for which the parameters are determined by previous optimisation process. The proposed approach is to find a small amount of local correction needed to satisfy given state constraints with the updated solution. Algorithms for updating the neural network parameters and the control function are considered.