Inference for batched bandits
File(s)
Accepted version
Author(s)
Zhang, Kelly W
Janson, Lucas
Murphy, Susan A
Type
Conference Paper
Abstract
As bandit algorithms are increasingly utilized in scientific studies and industrial applications, there is an associated increasing need for reliable inference methods based on the resulting adaptively-collected data. In this work, we develop methods for inference on data collected in batches using a bandit algorithm. We prove that the bandit arm selection probabilities cannot generally be assumed to concentrate. Non-concentration of the arm selection probabilities makes inference on adaptively-collected data challenging because classical statistical inference approaches, such as using asymptotic normality or the bootstrap, can have inflated Type-1 error and confidence intervals with below-nominal coverage probabilities even asymptotically. In response we develop the Batched Ordinary Least Squares estimator (BOLS) that we prove is (1) asymptotically normal on data collected from both multi-arm and contextual bandits and (2) robust to non-stationarity in the baseline reward and thus leads to reliable Type-1 error control and accurate confidence intervals.
Editor(s)
Larochelle, H
Ranzato, M
Hadsell, R
Balcan, MF
Lin, H
Date Issued
2020-12-12
Date Acceptance
2020-12-06
Citation
Advances in Neural Information Processing Systems, 2020, 33
ISSN
1049-5258
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Journal / Book Title
Advances in Neural Information Processing Systems
Volume
33
Copyright Statement
© 2020 Neural Information Processing Systems Foundation, Inc. (NeurIPS).
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/35002190
Source
34th Conference on Neural Information Processing Systems (NeurIPS)
Subjects
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Information Systems
MODELS
Science & Technology
Technology
Publication Status
Published
Start Date
2020-12-06
Finish Date
2020-12-12
Coverage Spatial
Online
Date Publish Online
2020-12-12