Context-based image acquisition

Abstract

The cost of off-chip memory access (in bandwidth, time and energy consumption) has become a major concern in the design of many hardware systems. Due to reasons such as the increasing performance gap between computing engines and memory systems, the process of data acquisition from memory has an increasingly dominant impact to the overall of the system performance. The cost in memory communication bandwidth and time consumption of data acquisition has become more significant, stalling the application and reducing its execution. Energy consumption has also become one of the main concerns to modern hardware systems, especially for embedded applications, and the energy spent on memory accessing has been reported to occupy a large proportion of the overall energy consumption of the system. All these lead to the research topic of reducing the cost of memory access in hardware systems.

Particularly for image processing systems, due to the ever growing size of image data, the task of image data acquisition poses an increasing challenge to the design of the systems. Various researches have addressed this problem of image data acquisition by exploiting the characteristics of memory structures and image processing applications. Some methods approach this problem from software perspective, changing for example the source code of the application so that the off-chip memory access is minimized; other methods approach this problem from hardware perspective, modifying the structure of memories and reorganizing the order of data transmission sequences. This thesis provides an alternative way of dealing with this problem and proposes the framework of ``Context-based Image Acquisition'' (CbIA) for hardware systems. Instead of accessing from the off-chip memory all image data requested by the application, the proposed framework accesses only fractions of the image and by utilizing image processing algorithms it reconstructs the missing part. This allows the proposed framework to trade computational effort for reduced cost of memory access, and ultimately trade image quality with reduced overall cost of the image acquisition process. On top of this, the proposed framework has the advantage of being independent from both the memory and the image processing application, and therefore can be seamlessly integrated into existing image processing systems.

The thesis elaborates on the proposed framework from both the algorithmic perspective and the hardware architectural perspective. A designed and implemented CbIA architecture is evaluated on reconfigurable hardware, reporting a reduction of up to 88\% of communication bandwidth, 68\% of time consumption, and 50\% of energy consumption of the image acquisition process, at the expense of reduced image quality (about 33 dB of PSNR on chosen benchmark images). Based on this design, this thesis also investigates on more complex algorithms at simulation level for CbIA procedures, including that for generic images as well as for specific image class. The use of more detailed modelling of images and/or domain-specific knowledge improves the ability of the CbIA procedure to trade image quality for bandwidth reduction (an increase of about 2 dB of PSNR with the same bandwidth used), with additional expenses on the computational cost. Finally, the thesis explores the impact brought by the proposed CbIA framework to the future development of hardware system.