Three-directional cine multi-slice left ventricular myocardial velocity mapping (3Dir MVM) is a cardiac magnetic resonance (CMR) technique that allows the assessment of cardiac motion in three orthogonal directions. Accurate and reproducible delineation of the myocardium is crucial for accurate analysis of peak systolic and diastolic myocardial velocities. In addition to the conventionally available magnitude CMR data, 3Dir MVM also provides three orthogonal phase velocity mapping datasets, which are used to generate velocity maps. These velocity maps may also be used to facilitate and improve the myocardial delineation. Based on the success of deep learning in medical image processing, we propose a novel fast and automated framework that improves the standard U-Net-based methods on these CMR multi-channel data (magnitude and phase velocity mapping) by cross-channel fusion with an attention module and the shape information-based post-processing to achieve accurate delineation of both epicardial and endocardial contours. To evaluate the results, we employ the widely used Dice Scores and the quantification of myocardial longitudinal peak velocities. Our proposed network trained with multi-channel data shows superior performance compared to standard U-Net-based networks trained on single-channel data. The obtained results are promising and provide compelling evidence for the design and application of our multi-channel image analysis of the 3Dir MVM CMR data.