Gait analysis plays an important role in several conditions, including the rehabilitation of patients with orthopaedic problems and the monitoring of neurological conditions, mental health problems and the well-being of elderly subjects. It also constitutes an index of good posture and thus it can be used to prevent injuries in athletes and monitor mental health in typical subjects. Usually, accurate gait analysis is based on the measurement of ankle dynamics and ground reaction forces. Therefore, it requires expensive multi-camera systems and pressure sensors, which cannot be easily employed in a free-living environment. We propose a fusion framework that uses an ear worn activity recognition (e-AR) sensor and a single video camera to estimate foot angle during key gait events. To this end we use canonical correlation analysis with a fused-lasso penalty in a two-steps approach that firstly learns a model of the timing distribution of ground reaction forces based on e-AR signal only and subsequently models the eversion/inversion as well as the dorsiflexion of the ankle based on the combined features of e-AR sensor and the video. The results show that incorporating invariant features of angular ankle information from the video recordings improves the estimation of the foot progression angle, substantially.