Distributed testing and estimation in sparse high dimensional models

File Description SizeFormat 
aosAccepted.pdfAccepted version1.62 MBAdobe PDFView/Open
Title: Distributed testing and estimation in sparse high dimensional models
Authors: Battey, HS
Zhu, Z
Fan, J
Lu, J
Liu, H
Item Type: Journal Article
Abstract: This paper studies hypothesis testing and parameter estimation in the context of the divide-and-conquer algorithm. In a unified likelihood-based framework, we propose new test statistics and point estimators obtained by aggregating various statistics from k subsamples of size n/k, where n is the sample size. In both low dimensional and sparse high dimensional settings, we address the important question of how large k can be, as n grows large, such that the loss of efficiency due to the divide-and-conquer algorithm is negligible. In other words, the resulting estimators have the same inferential efficiencies and estimation rates as an oracle with access to the full sample. Thorough numerical results are provided to back up the theory.
Issue Date: 3-May-2018
Date of Acceptance: 17-May-2017
URI: http://hdl.handle.net/10044/1/48600
DOI: https://dx.doi.org/10.1214/17-AOS1587
ISSN: 0090-5364
Publisher: Institute of Mathematical Statistics
Start Page: 1352
End Page: 1382
Journal / Book Title: Annals of Statistics
Volume: 46
Issue: 3
Copyright Statement: © Institute of Mathematical Statistics, 2018
Keywords: 0104 Statistics
1403 Econometrics
Statistics & Probability
Publication Status: Published
Appears in Collections:Mathematics
Faculty of Natural Sciences

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Creative Commons