Continuous monitoring for changepoints in data streams using adaptive estimation
File(s)supp_continuousmonitoring.pdf (1005.88 KB) main_continuousmonitoring.pdf (497.24 KB)
Supporting information
Accepted version
Author(s)
Bodenham, DA
Adams, NM
Type
Journal Article
Abstract
Data streams are characterised by a potentially unending sequence of high-frequency observations which are subject to unknown temporal variation. Many modern streaming applications demand the capability to sequentially detect changes as soon as possible after they occur, while continuing to monitor the stream as it evolves. We refer to this problem as continuous monitoring. Sequential algorithms such as CUSUM, EWMA and their more sophisticated variants usually require a pair of parameters to be selected for practical application. However, the choice of parameter values is often based on the anticipated size of the changes and a given choice is unlikely to be optimal for the multiple change sizes which are likely to occur in a streaming data context. To address this critical issue, we introduce a changepoint detection framework based on adaptive forgetting factors that, instead of multiple control parameters, only requires a single parameter to be selected. Simulated results demonstrate that this framework has utility in a continuous monitoring setting. In particular, it reduces the burden of selecting parameters in advance. Moreover, the methodology is demonstrated on real data arising from Foreign Exchange markets.
Date Issued
2017-09-01
Date Acceptance
2016-07-08
Citation
Statistics and Computing, 2017, 27 (5), pp.1257-1270
ISSN
0960-3174
Publisher
Springer
Start Page
1257
End Page
1270
Journal / Book Title
Statistics and Computing
Volume
27
Issue
5
Copyright Statement
© Springer Science+Business Media New York 2016. The final publication is available at Springer via https://link.springer.com/article/10.1007%2Fs11222-016-9684-8
Subjects
Science & Technology
Technology
Physical Sciences
Computer Science, Theory & Methods
Statistics & Probability
Computer Science
Mathematics
Changepoint detection
Adaptive estimation
Data stream
Sequential analysis
CONTROL CHARTS
0104 Statistics
0802 Computation Theory and Mathematics
Statistics & Probability
Publication Status
Published
Date Publish Online
2016-07-18