Automated quality control in image segmentation: application to the UK Biobank cardiac MR imaging study

File Description SizeFormat 
Robinson_Automated quality control in image_BMC.pdfPublished version3.4 MBAdobe PDFView/Open
Title: Automated quality control in image segmentation: application to the UK Biobank cardiac MR imaging study
Authors: Robinson, R
Valindria, VV
Bai, W
Oktay, O
Kainz, B
Suzuki, H
Sanghvi, MM
Aung, N
Paiva, JÉM
Zemrak, F
Fung, K
Lukaschuk, E
Lee, AM
Carapella, V
Kim, YJ
Piechnik, SK
Neubauer, S
Petersen, SE
Page, C
Matthews, PM
Rueckert, D
Glocker, B
Item Type: Journal Article
Abstract: Background: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools, e.g. image segmentation methods, are employed to derive quantitative measures or biomarkers for later analyses. Manual inspection and visual QC of each segmentation isn't feasible at large scale. However, it's important to be able to automatically detect when a segmentation method fails so as to avoid inclusion of wrong measurements into subsequent analyses which could lead to incorrect conclusions. Methods: To overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4,800 cardiac magnetic resonance scans. We then apply our method to a large cohort of 7,250 cardiac MRI on which we have performed manual QC. Results: We report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4,800 scans for which manual segmentations were available. We mimic real-world application of the method on 7,250 cardiac MRI where we show good agreement between predicted quality metrics and manual visual QC scores. Conclusions: We show that RCA has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study.
Issue Date: 14-Mar-2019
Date of Acceptance: 3-Feb-2019
ISSN: 1097-6647
Publisher: BioMed Central
Journal / Book Title: Journal of Cardiovascular Magnetic Resonance
Volume: 21
Replaces: 10044/1/67178
Copyright Statement: © The Author(s). 2019Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0International License (, which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to theCreative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver( applies to the data made available in this article, unless otherwise stated.
Sponsor/Funder: Engineering & Physical Science Research Council (EPSRC)
Engineering & Physical Science Research Council (EPSRC)
National Institute for Health Research
Commission of the European Communities
Innovate UK
Funder's Grant Number: EP/N014529/1
H2020 - 757173
Keywords: Automatic quality control
Population imaging
1102 Cardiovascular Medicine And Haematology
Nuclear Medicine & Medical Imaging
Notes: 14 pages, 7 figures, Journal of Cardiovascular Magnetic Resonance
Publication Status: Published
Open Access location:
Article Number: ARTN 18
Appears in Collections:Computing
Department of Medicine (up to 2019)
Faculty of Medicine

Unless otherwise indicated, items in Spiral are protected by copyright and are licensed under a Creative Commons Attribution NonCommercial NoDerivatives License.

Creative Commons