Deep learning models for semantic segmentation are able to learn powerful
representations for pixel-wise predictions, but are sensitive to noise at test
time and do not guarantee a plausible topology. Image registration models on
the other hand are able to warp known topologies to target images as a means of
segmentation, but typically require large amounts of training data, and have
not widely been benchmarked against pixel-wise segmentation models. We propose
Atlas-ISTN, a framework that jointly learns segmentation and registration on 2D
and 3D image data, and constructs a population-derived atlas in the process.
Atlas-ISTN learns to segment multiple structures of interest and to register
the constructed, topologically consistent atlas labelmap to an intermediate
pixel-wise segmentation. Additionally, Atlas-ISTN allows for test time
refinement of the model's parameters to optimize the alignment of the atlas
labelmap to an intermediate pixel-wise segmentation. This process both
mitigates for noise in the target image that can result in spurious pixel-wise
predictions, as well as improves upon the one-pass prediction of the model.
Benefits of the Atlas-ISTN framework are demonstrated qualitatively and
quantitatively on 2D synthetic data and 3D cardiac computed tomography and
brain magnetic resonance image data, out-performing both segmentation and
registration baseline models. Atlas-ISTN also provides inter-subject
correspondence of the structures of interest, enabling population-level shape
and motion analysis.