Deep learning using convolutional neural networks in clinical cardiology

Abstract

Computers have revolutionised modern medicine over the last two decades, with the advent of electronic health records and digital imaging. However, it has been difficult to develop computer programs which can assist doctors in interpreting these data. The traditional approach of programming a computer with an algorithm works well for simple decision making on low-dimensional data such as a series of blood tests. However, for highly complex data such as X-rays, echocardiography and magnetic resonance imaging scans, it has seemed impossible to develop adequately performing algorithms for even the simplest medical tasks. Recently, however, a new paradigm in computer programming has emerged: machine learning. In machine learning, a computer learns how to perform a task without being explicitly programmed how to do it, and an increasingly popular approach to machine learning is ‘deep learning’ with neural networks. These neural networks somewhat resemble the animal brain in that they are made up of a series of neurons with adjustable connections between them. These connections are akin to synapses, and by tuning them optimally, computers can perform complex tasks such as image interpretation. In this thesis I explore applications of neural networks to provide automated assistance for common cardiac image interpretation tasks: • identifying the make and model of a cardiac device on a chest X-ray • distinguishing successful from unsuccessful His bundle pacing, using an ECG • identifying which view is represented on an echocardiogram video loop • rapidly interpret the early anatomy sequences on a cardiac magnetic resonance (CMR) scan For each of these tasks, I developed a specialised neural network design. They are all ‘convolutional neural networks’, which are inspired by the mammalian visual cortex, but they have different structures which I have selected to handle the different tasks. First, I develop a convolutional neural network which can identify the exact model of pacemaker or defibrillator a patient has from their chest X-ray. The network is so effective that its performance exceeds that of cardiac electrophysiologists performing the same task. This system could speed up the diagnosis and treatment of patients with cardiac rhythm devices. After I published it and made it available online, doctors have reported using it successfully in their clinical practice. Second, I develop a convolutional neural network which can discriminate between the ECG responses to His bundle pacing. It is able to identify when the procedure has been successful and is able to perform this task without requiring the intra-procedural electrograms and complex pacing manoeuvres that humans rely on to perform this task. Third, I develop a series of neural networks which can classify echocardiogram videos according to what anatomical structures are depicted in them. This novel approach to ‘view classification’ more than halves the error rate of previous state-of-the-art methods. Finally, I investigate whether deep learning of the anatomy sequences acquired in the first minutes of a cardiac magnetic resonance scan could provide early useful diagnostic information. I train a convolutional neural network to reconstruct a 3-dimensional model of the heart and major vessels from these early images and find it was able to accurately quantify cardiac chamber size, aortic diameter and identify the presence of pleural effusions. This system could be useful for identifying unexpected pathology, to allow optimisation of the scanning protocol within minutes of a scan commencing.