A Low-power Neuromorphic System for Retinal Implants and Sensory Substitution

Abstract

This paper describes the design and operation of a system which can be used as a Visual to Auditory Sensory Substitution Device (SSD), as well as the front-end of a real-time retinal prosthesis (RP) or Vision Augmentation (VA) system. Such systems consist of three components: a sensory block to capture the visual scene, a processing block to manage the collected data and generate stimulus patterns, and an output block. For the sensory block we use a Dynamic Vision Sensor (DVS) instead of a conventional camera. A microcontroller is used as the processing block, which receives asynchronous inputs from the DVS in the form of ON/OFF events and treats them like post-synaptic potentials. A simple algorithm based on an Integrate & Fire neuron model is used to emulate temporal contrast sensitive Retinal Ganglion Cells (RGCs). For an RP system the output would be an implanted electrode array, whereas for the SSD a sound is activated based on a certain mapping algorithm. The results are shown in the form of ON or OFF events on the LED matrix (equivalent to the stimulation pattern on an electrode array in the case of an RP), and in the form of a stereo sound output.