Cox reduction and confidence sets of models: a theoretical elucidation
File(s)
Accepted version
Author(s)
Lewis, Rebecca
Battey, Heather
Type
Journal Article
Abstract
For sparse high-dimensional regression problems, Cox and Battey [2, 14] emphasised the need for confidence sets of models: an enumeration of those small sets of variables that fit the data equivalently well in a suitable statistical sense. This is to be contrasted with the single model returned by penalised regression procedures, effective for prediction but potentially misleading for subject-matter understanding. The proposed construction of such sets relied on preliminary reduction of the full set of variables. While various possibilities could be considered for this, [14] proposed a succession of regression fits based on incomplete block designs. The purpose of the present paper is to provide insight on both aspects of [14]. For an unspecified reduction strategy, we begin by characterising models that are likely to be retained in the model confidence set, emphasising geometric aspects. We then evaluate possible reduction schemes based on penalised regression or marginal screening, before theoretically elucidating the reduction of [14]. We identify features of the covariate matrix that may reduce its efficacy, and indicate improvements to the original proposal. An advantage of the approach is its ability to reveal its own stability or fragility for the data at hand.
Date Issued
2024
Date Acceptance
2024-04-21
Citation
Statistical Science, 2024
ISSN
0883-4237
Publisher
Institute of Mathematical Statistics
Journal / Book Title
Statistical Science
Copyright Statement
Subject to copyright. This paper is embargoed until publication. Once published the author’s accepted manuscript will be made available under a CC-BY License in accordance with Imperial’s Research Publications Open Access policy (www.imperial.ac.uk/oa-policy).
Publication Status
Accepted
Rights Embargo Date
10000-01-01