Data-driven spatial sparsity for convolutional neural networks
File(s)
Author(s)
Giorgos, Zampokas
Type
Thesis
Abstract
Advancements in Convolutional Neural Networks (CNNs) have driven significant research efforts towards optimizing their effectiveness and applicability. A promising yet less explored path lies in leveraging spatially adaptive computation within networks—an approach aimed at reducing computation redundancy and bolstering overall efficiency. This work investigates the presence of computational redundancy within CNN computations in vision tasks, and explores dynamic optimization methodologies to increase model efficiency.
Firstly, the thesis explores the balance between accuracy and computational cost in stereo matching CNNs. Employing a data-driven strategy, it seeks to control this trade-off by harnessing spatially adaptive computation to minimize redundant computations effectively. By skipping computations in specific locations, our findings indicate that the efficiency of stereo-matching CNNs can be improved while maintaining accuracy.
Expanding the scope, the work tackles the optimization of semantic segmentation CNNs, focusing on either Floating Point Operations (FLOPs) or practical latency. This is achieved through a data-driven optimization methodology which leverages spatially adaptive processing, aiming to meet a specified target budget. When considering practical latency, execution time measurements from target devices are introduced to guide the optimization process.
Lastly, the multifaceted challenge of performing inference at multiple computational budgets while considering the memory footprint of the model weights is explored. Employing spatially adaptive dynamic execution methods, the goal is to strike a balance between memory constraints and computational efficiency, addressing the challenge of optimizing models within restricted memory environments, when multiple performance points are required.
In conclusion, this study explores optimization of CNNs for vision tasks including stereo matching, semantic segmentation and image classification, probing the potential to balance accuracy against multiple axes by considering FLOPs, latency, and memory. It aims to leverage spatially adaptive computation, dynamic data-driven techniques, and novel architectures to enhance efficiency, offering a holistic approach that navigates the trade-offs and boundaries of neural network optimization.
Firstly, the thesis explores the balance between accuracy and computational cost in stereo matching CNNs. Employing a data-driven strategy, it seeks to control this trade-off by harnessing spatially adaptive computation to minimize redundant computations effectively. By skipping computations in specific locations, our findings indicate that the efficiency of stereo-matching CNNs can be improved while maintaining accuracy.
Expanding the scope, the work tackles the optimization of semantic segmentation CNNs, focusing on either Floating Point Operations (FLOPs) or practical latency. This is achieved through a data-driven optimization methodology which leverages spatially adaptive processing, aiming to meet a specified target budget. When considering practical latency, execution time measurements from target devices are introduced to guide the optimization process.
Lastly, the multifaceted challenge of performing inference at multiple computational budgets while considering the memory footprint of the model weights is explored. Employing spatially adaptive dynamic execution methods, the goal is to strike a balance between memory constraints and computational efficiency, addressing the challenge of optimizing models within restricted memory environments, when multiple performance points are required.
In conclusion, this study explores optimization of CNNs for vision tasks including stereo matching, semantic segmentation and image classification, probing the potential to balance accuracy against multiple axes by considering FLOPs, latency, and memory. It aims to leverage spatially adaptive computation, dynamic data-driven techniques, and novel architectures to enhance efficiency, offering a holistic approach that navigates the trade-offs and boundaries of neural network optimization.
Version
Open Access
Date Issued
2024-05-24
Date Awarded
01/04/2025
License URL
Advisor
Christos-Savvas, Bouganis
Publisher Department
Department of Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)