Harnessing reconfigurable hardware capabilities for 3D CNN acceleration
File(s)
Author(s)
Toupas, Petros
Type
Thesis
Abstract
As computer vision research shifts towards video and volumetric data, the demand for efficient processing of spatiotemporal and multi-dimensional data has grown significantly. This thesis investigates the optimisation and mapping of 3D Convolutional Neural Networks (CNNs) onto Field-Programmable Gate Arrays (FPGAs), addressing the growing demand for hardware acceleration in applications like human action recognition (HAR), and medical image segmentation. Compared to 2D CNNs, 3D CNNs introduce additional complexity, larger workloads and higher memory demands. The research explores novel FPGA-based methodologies for both throughput- and latency-optimized designs to address these challenges.
Synchronous dataflow is utilised to generate intermediate representations of 3D CNNs, facilitating transformations to expand the design space. A key contribution is the development of automated toolflows for mapping and optimising 3D CNNs onto FPGA devices, addressing the lack of model-agnostic solutions. Adaptive optimization strategies are explored to automatically adjust to varying model architectures and hardware constraints. The research emphasizes on balancing between on-chip and off-chip memory utilisation, as the large data volumes of 3D models often exceed on-chip memory resources requiring off-chip access that can degrade performance. Strategies to address these challenges are investigated, particularly in streaming architectures featuring complex blocks like residual connections and branching. Another key contribution is the introduction of novel design space exploration and performance modelling methods, for efficiently navigating the 3D CNNs' broadened design space. The study additionally investigates runtime parametrisation and reconfigurable computing techniques to improve the flexibility and efficiency of the proposed accelerators. The findings demonstrate significant improvements in both throughput and latency oriented architectures across several 3D CNN models, highlighting the efficiency of the proposed solutions. While primarily focused on Human Action Recognition (HAR), the methodologies of this thesis are broadly applicable to other domains, like video analysis and medical imaging, enabling the 3D CNN implementation and deployment in resource-constrained settings.
Synchronous dataflow is utilised to generate intermediate representations of 3D CNNs, facilitating transformations to expand the design space. A key contribution is the development of automated toolflows for mapping and optimising 3D CNNs onto FPGA devices, addressing the lack of model-agnostic solutions. Adaptive optimization strategies are explored to automatically adjust to varying model architectures and hardware constraints. The research emphasizes on balancing between on-chip and off-chip memory utilisation, as the large data volumes of 3D models often exceed on-chip memory resources requiring off-chip access that can degrade performance. Strategies to address these challenges are investigated, particularly in streaming architectures featuring complex blocks like residual connections and branching. Another key contribution is the introduction of novel design space exploration and performance modelling methods, for efficiently navigating the 3D CNNs' broadened design space. The study additionally investigates runtime parametrisation and reconfigurable computing techniques to improve the flexibility and efficiency of the proposed accelerators. The findings demonstrate significant improvements in both throughput and latency oriented architectures across several 3D CNN models, highlighting the efficiency of the proposed solutions. While primarily focused on Human Action Recognition (HAR), the methodologies of this thesis are broadly applicable to other domains, like video analysis and medical imaging, enabling the 3D CNN implementation and deployment in resource-constrained settings.
Version
Open Access
Date Issued
2024-12-07
Date Awarded
01/10/2025
License URL
Advisor
Bouganis, Christos-Savvas
Tzovaras, Dimitrios
Publisher Department
Department of Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)