Metalearning-based alternating minimization algorithm for nonconvex optimization
File(s)XLHYJG_TNNLS22.pdf (7.99 MB)
Accepted version
Author(s)
Type
Journal Article
Abstract
In this article, we propose a novel solution for nonconvex problems of multiple variables, especially for those typically solved by an alternating minimization (AM) strategy that splits the original optimization problem into a set of subproblems corresponding to each variable and then iteratively optimizes each subproblem using a fixed updating rule. However, due to the intrinsic nonconvexity of the original optimization problem, the optimization can be trapped into a spurious local minimum even when each subproblem can be optimally solved at each iteration. Meanwhile, learning-based approaches, such as deep unfolding algorithms, have gained popularity for nonconvex optimization; however, they are highly limited by the availability of labeled data and insufficient explainability. To tackle these issues, we propose a meta-learning based alternating minimization (MLAM) method that aims to minimize a part of the global losses over iterations instead of carrying minimization on each subproblem, and it tends to learn an adaptive strategy to replace the handcrafted counterpart resulting in advance on superior performance. The proposed MLAM maintains the original algorithmic principle, providing certain interpretability. We evaluate the proposed method on two representative problems, namely, bilinear inverse problem: matrix completion and nonlinear problem: Gaussian mixture models. The experimental results validate the proposed approach outperforms AM-based methods.
Date Issued
2023-09-01
Date Acceptance
2022-04-01
Citation
IEEE Transactions on Neural Networks and Learning Systems, 2023, 34 (9), pp.5366-5380
ISSN
1045-9227
Publisher
Institute of Electrical and Electronics Engineers
Start Page
5366
End Page
5380
Journal / Book Title
IEEE Transactions on Neural Networks and Learning Systems
Volume
34
Issue
9
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000785812200001&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Subjects
Alternating minimization (AM)
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Hardware & Architecture
Computer Science, Theory & Methods
DECONVOLUTION
Deep learning
deep unfolding
Engineering
Engineering, Electrical & Electronic
gaussian mixture model (GMM)
Iterative algorithms
matrix completion
MATRIX COMPLETION
MAXIMUM-LIKELIHOOD
meta-learning (ML)
Minimization
MODEL
Neural networks
Optimization
Science & Technology
Signal processing algorithms
Task analysis
Technology
Publication Status
Published
Date Publish Online
2022-04-19