A machine learning approach to accelerate convergence of the nonlinear solver in multiphase flow problems is presented here. The approach dynamically controls an acceleration method based on numerical relaxation. It is demonstrated in a Picard iterative solver but is applicable to other types of nonlinear solvers. The aim of the machine learning acceleration is to reduce the computational cost of the nonlinear solver by adjusting to the complexity/physics of the system. Using dimensionless parameters to train and control the machine learning enables the use of a simple two-dimensional layered reservoir for training, while also exploring a wide range of the parameter space. Hence, the training process is simplified and it does not need to be rerun when the machine learning acceleration is applied to other reservoir models. We show that the method can significantly reduce the number of nonlinear iterations without compromising the simulation results, including models that are considerably more complex than the training case.