Medical imaging plays a crucial role in modern medicine, where it aims to provide practitioners visual guidance to assess and diagnose underlying medical conditions of patients. In medical image reconstruction, the aim is to provide high-quality and clinically accurate images of the anatomy for clinical use at minimal costs and risks to the patients. Early reconstruction methods were mostly designed for a specific target application based on a hypothesis and/or prior of the image to be reconstructed, commonly known as hand-crafted models, e.g., atlas-based segmentation of MR volumes. With modern advances in technology, especially in deep learning methods, this has caused a paradigm shift where models are now mostly data-driven. Deep learning models are provided with a large cohort of carefully curated image and target pairs, known as a dataset. An optimisation algorithm then finds the closest matching mapping function that maps the input to its specific target. Image synthesis extends beyond the scope of reconstruction, where the task is to generate new samples that closely resemble the dataset. Both hand-crafted and data-driven modelling have their own advantages and disadvantages, which shall be explored in this thesis.
This thesis explores both applications of reconstruction and synthesis in medical imaging using deep learning. For medical image reconstruction, a new method is proposed to perform 2D to 3D image reconstruction with motion correction in fetal brain MRI using Deep Learning. Scanned images often suffer from inter-slice motion artefacts due to unconstrained fetal movements during examination. This can cause neighbouring slices to become incoherent and the 3D volume corrupt. Traditional intensity-based 2D to 3D reconstruction algorithms are only effective if there are little motion, thus said to have a small capture range. Excessive motion usually means reconstruction is not possible, and a second scan would have to be arranged, which is both costly and time consuming. The contribution of this thesis is two-fold for fetal brain MRI: the problem of 2D to 3D reconstruction is analysed in depth, with a dataset specifically curated from existing 3D reconstructed brains, to allow for 2D to 3D reconstruction with infinite capture range. This method is then further extended,with a specifically designed loss function based on Riemannian geometry, to better constrain the optimisation process during training.
For medical image synthesis, the inverse problem of 2D to 3D registration is explored. Given a particular pose parameter, and a few patient specific 2D slices of a fetal brain as context, a model is designed to synthesise images at particular view locations for that particular patient. This greatly allows for reconstruction of whole 3D volumes from fast, sparsely sampled 2D slices. Two further applications were explored as part of medical image synthesis, other than fetal brain MRI, to show general applicability: DRR-to-CT reconstruction and fundus image synthesis in diabetic retinopathy for dataset augmentation.
The methods for fetal brain MRI are evaluated on the Intelligent Fetal Image and Diagnostic (iFIND) dataset, as well as the Developing Human Connectome Project (dHCP) dataset. For DRR-to-CT reconstruction, a publically available CT dataset was used along with Siddon-Jacobs raytracing algorithms to create paired CT and synthetic Digital Reconstructed Radiographs (DRR) X-ray images. Finally for Fundus image synthesis in diabetic retinopathy, the kaggle diabetic retinopathy dataset from EyePACS was used. Each application/model were assessed by quantitative and qualitative evaluation techniques commonly used in the literature to compare and contrast the
effectiveness of different approaches.