Performance metrics for characterization of a seizure detection algorithm for offline and online use

Abstract

Purpose: To select appropriate previously reported performance metrics to evaluate a new seizure detection algorithm for offline and online analysis, and thus quantify any performance variation between these metrics. Methods: Traditional offline algorithms mark out any EEG section (epoch) of a seizure (event), so that neurologists only analyze the detected and adjacent sections. Thus, offline algorithms could be evaluated using number of correctly detected events, or event-based sensitivity (SEVENT), and epoch-based specificity (percentage of incorrectly detected background epochs). In contrast, online seizure detection (especially, data selection) algorithms select for transmission only the detected EEG sections and hence need to detect the entire duration of a seizure. Thus, online algorithms could be evaluated using percentage of correctly detected seizure duration, or epoch-based sensitivity (SEPOCH), and epoch-based specificity. Here, a new seizure detection algorithm is evaluated using the selected performance metrics for epoch duration ranging from 1s to 60s. Results: For 1s epochs, the area under the event-based sensitivity-specificity curve was 0.95 whilst SEPOCH achieves 0.81. This difference is not surprising, as intuitively, detecting any epoch within a seizure is easier than detecting every epoch - especially as seizures evolve over time. For longer epochs of 30s or 60s, SEVENT falls to 0.84 and 0.82 respectively and SEPOCH reduces to 0.76. Here, decreased SEVENT shows that fewer seizures are detected, possibly due to easy-to-detect short seizure sections being masked by surrounding EEG. However, detecting one long epoch constitutes a larger percentage of a seizure than a shorter one and thus SEPOCH does not decrease proportionately. Conclusions: Traditional offline and online seizure detection algorithms require different metrics to effectively evaluate their performance for their respective applications. Using such metrics, it has been shown that a decrease in performance may be expected when an offline seizure detection algorithm (especially with short epoch duration) is used for online analysis.