Precise and objective segmentation of atrial scarring (SAS) is a prerequisite for quantitative assessment of atrial fibrillation using non-invasive late gadolinium-enhanced (LGE) MRI. This also requires accurate delineation of the left atrium (LA) and pulmonary veins (PVs) geometry. Most previous studies have relied on manual segmentation of LA wall and PVs, which is a tedious and error-prone procedure with limited reproducibility. There are many attempts on automatic SAS using simple thresholding, histogram analysis, clustering and graph-cut based approaches; however, in general, these methods are considered as unsupervised learning thus subject to limited segmentation accuracy. In this study, we present a fully-automated multi-atlas based whole heart segmentation method to derive the LA and PVs geometry objectively that is followed by a fully automatic deep learning method for SAS. Our deep learning method consists of a feature extraction step via super-pixel over-segmentation and a supervised classification step via stacked sparse auto-encoders. We demonstrate the efficacy of our method on 20 clinical LGE MRI scans acquired from a longstanding persistent atrial fibrillation cohort. Both quantitative and qualitative results show that our fully automatic method obtained accurate segmentation results compared to the manual segmentation based ground truths.