Robust solutions to constrained optimization problems by LSTM networks

Abstract

Many technical issues for communications and computer infrastructures, including resource sharing, network management and distributed analytics, can be formulated as optimization problems. Gradient-based iterative algorithms have been widely utilized to solve these problems. Much research focuses on improving the iteration convergence. However, when system parameters change, it requires a new solution from the iterative methods. Therefore, it is helpful to develop machine-learning solution frameworks that can quickly produce solutions over a range of system parameters. We propose here a learning approach to solve non-convex, constrained optimization problems. Two coupled Long Short Term Memory (LSTM) networks are used to find the optimal solution. The advantages of this new framework include: (1) near optimal solution for a given problem instance can be obtained in very few iterations (time steps) during the inference process, (2) the learning approach allows selections of various hyper-parameters to achieve desirable tradeoffs between the training time and the solution quality, and (3) the coupled-LSTM networks can be trained using system parameters with distributions different from those used during inference to generate solutions, thus enhancing the robustness of the learning technique. Numerical experiments using a dataset from Alibaba reveal that the relative discrepancy between the generated solution and the optimum is less than 1% and 0.1% after 2 and 12 iterations, respectively.