Event-based sensing is a bio-inspired sensing scheme where the system responds to changes in the input stimuli and generates a stream of events. Event-based sensors, especially event cameras, have many attractive properties, including high temporal resolution, low latency, high dynamic range and low energy consumption. Event cameras have great potential to be widely used in various computer vision applications. In this thesis, we explore machine learning techniques to achieve high-quality events-to-video reconstruction, improve video-to-events generation, and develop efficient schemes for classifying event-based signals.

First, we introduce a family of model-based deep networks for high-quality events-to-video reconstruction. A convolutional ISTA network is designed based on sparse representation models and algorithm unfolding strategy. Different temporal consistency constraints are further introduced to enhance the temporal coherence between frames. Finally, we integrate optical flow estimation into the reconstruction network for motion compensation. Results highlight that our networks outperform state-of-the-art methods and achieve better temporal consistency.

Next, we explore ways to improve video-to-events generation. We introduce the idea of having interleaved pixels with different acquisition factors to extract more useful information from intensity. We also propose incorporating adaptive temporal oscillations into intensity changes to amplify small changes. Results show that diversity and oscillations in neuromorphic sensing reveals more fine details and this leads to improved reconstruction quality.

Finally, we focus on the classification problem for event sequences with complex temporal structures using spiking neural networks (SNNs). We present a novel decision-making strategy based on first-spike (FS) coding and propose a surrogate gradient learning method. Furthermore, we propose a current-based adaptive leaky integrate-and-fire neuron to enhance the capability of SNNs in processing rich temporal information. Results show that FS coding achieves comparable accuracy to firing rate coding while leading to superior energy efficiency and distinct neuronal dynamics on data sequences with very rich temporal structures.