Adaptation and learning over networks under subspace constraints-part I: stability analysis
OA Location
Author(s)
Nassif, Roula
Vlaski, Stefan
Sayed, Ali H
Type
Journal Article
Abstract
This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in low-dimensional subspaces. This constrained formulation includes consensus optimization as a special case, and allows for more general task relatedness models such as smoothness. While such formulations can be solved via projected gradient descent, the resulting algorithm is not distributed. Starting from the centralized solution, we propose an iterative and distributed implementation of the projection step, which runs in parallel with the stochastic gradient descent update. We establish in this Part I of the work that, for small step-sizes μ, the proposed distributed adaptive strategy leads to small estimation errors on the order of μ. We examine in the accompanying Part II (R. Nassif, S. Vlaski, and A. H. Sayed, 2019) the steady-state performance. The results will reveal explicitly the influence of the gradient noise, data characteristics, and subspace constraints, on the network performance. The results will also show that in the small step-size regime, the iterates generated by the distributed algorithm achieve the centralized steady-state performance.
Date Issued
2020
Date Acceptance
2020-01-22
Citation
IEEE Transactions on Signal Processing, 2020, 68, pp.1346-1360
ISSN
1053-587X
Publisher
Institute of Electrical and Electronics Engineers
Start Page
1346
End Page
1360
Journal / Book Title
IEEE Transactions on Signal Processing
Volume
68
Copyright Statement
© 2020 IEEE
Identifier
https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000526718500007&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=a2bf6146997ec60c407a63945d4e92bb
Subjects
ALGORITHMS
BEAMFORMER
Distributed optimization
Engineering
Engineering, Electrical & Electronic
gradient noise
LMS
Science & Technology
SENSOR NETWORKS
stability analysis
subspace projection
Technology
Publication Status
Published
Date Publish Online
2020-01-29