Scoring predictions at extreme quantiles
File(s)manuscript_acceptedon30Sep2021.pdf (362.89 KB)
Accepted version
Author(s)
Gandy, Axel
Jana, Kaushik
Veraart, Almut
Type
Journal Article
Abstract
Prediction of quantiles at extreme tails is of interest in numerous
applications. Extreme value modelling provides various competing predictors
for this point prediction problem. A common method of assessment of a set
of competing predictors is to evaluate their predictive performance in a given
situation. However, due to the extreme nature of this inference problem, it can
be possible that the predicted quantiles are not seen in the historical records,
particularly when the sample size is small. This situation poses a problem to
the validation of the prediction with its realisation. In this article, we propose
two non-parametric scoring approaches to assess extreme quantile prediction
mechanisms. The proposed assessment methods are based on predicting a sequence of equally extreme quantiles on different parts of the data. We then
use the quantile scoring function to evaluate the competing predictors. The
performance of the scoring methods is compared with the conventional scoring method and the superiority of the former methods are demonstrated in a
simulation study. The methods are then applied to reanalyse cyber Netflow
data from Los Alamos National Laboratory and daily precipitation data at a
station in California available from Global Historical Climatology Network.
applications. Extreme value modelling provides various competing predictors
for this point prediction problem. A common method of assessment of a set
of competing predictors is to evaluate their predictive performance in a given
situation. However, due to the extreme nature of this inference problem, it can
be possible that the predicted quantiles are not seen in the historical records,
particularly when the sample size is small. This situation poses a problem to
the validation of the prediction with its realisation. In this article, we propose
two non-parametric scoring approaches to assess extreme quantile prediction
mechanisms. The proposed assessment methods are based on predicting a sequence of equally extreme quantiles on different parts of the data. We then
use the quantile scoring function to evaluate the competing predictors. The
performance of the scoring methods is compared with the conventional scoring method and the superiority of the former methods are demonstrated in a
simulation study. The methods are then applied to reanalyse cyber Netflow
data from Los Alamos National Laboratory and daily precipitation data at a
station in California available from Global Historical Climatology Network.
Date Issued
2022-12-01
Date Acceptance
2021-09-29
Citation
AStA Advances in Statistical Analysis, 2022, 106, pp.527-544
ISSN
0002-6018
Publisher
Springer
Start Page
527
End Page
544
Journal / Book Title
AStA Advances in Statistical Analysis
Volume
106
Copyright Statement
© Springer-Verlag GmbH Germany, part of Springer Nature 2022. The final publication is available at Springer via https://doi.org/10.1007/s10182-021-00421-9
Sponsor
Lloyd's Register Foundation
Identifier
https://link.springer.com/article/10.1007/s10182-021-00421-9
Grant Number
ATIPO000005051
Subjects
Science & Technology
Physical Sciences
Statistics & Probability
Mathematics
Extreme value
High quantile
Quantile score
REGRESSION
SELECTION
stat.AP
stat.AP
60G70
0104 Statistics
Economics
Publication Status
Published
Date Publish Online
2022-02-14