Neural information storage and processing in the presence of noise and nonlinearity
Author(s)
El-Geresy, Waleed
Type
Thesis
Abstract
Artificial intelligence has grown exponentially in popularity in recent years, thanks to impressive applications, especially in the field of generative modelling in visual and linguistic domains. A multitude of advancements in understanding of the theory of intelligence and of artificial neural networks have been made, which have drawn modern neural network architectures and algorithms further and further from their origins as models of networks of biological neurons in the human brain. This thesis argues that since the human brain is the source of our definition of intelligence, furthering our understanding of its computational methods will be the key to unbridling the field of AI from the reins of its huge energy costs, and bridging the gap between algorithms and the physical world. To this end, we explore a number of computational primitives present in biological neurons and neural networks, looking at the benefits, challenges and limitations of implementing them artificially, and arguing the case for an approach to AI research with awareness and exploitation of noise and non-ideality as fundamental components. To this end, we take a particular focus on devices called memristors, which have been proposed as passive electronic components that could be used for neuro-inspired computing. We investigate the ability of memristors to implement neuronal computational primitives, and address the problem of noisy information storage on the devices in the presence of resistive drift noise, using joint source-channel coding. We further explore the notion of intelligent processing in the presence of noise through identifying a use case for adversarial noise, generated as part of a competitive minimax communication game, exploring how it can be used to enable distribution matching in a generative modelling problem.
Date Issued
2024-03-13
Date Awarded
01/03/2025
License URL
Advisor
Gündüz, Deniz
Papavassiliou, Christos
Sponsor
Engineering and Physical Sciences Research Council
Grant Number
EP/R513052/1
EP/N509486/1
Publisher Department
Department of Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)