Cost-effective solution to synchronised audio-visual data capture using multiple sensors
File(s)
Published version
Author(s)
Lichtenauer, JF
Shen, J
Valstar, MF
Pantic, Maja
Type
Report
Abstract
Applications such as surveillance and human behaviour analysis require high-
bandwidth recording from multiple cameras, as well as from other sensors. In
turn, sensor fusion has increased the required accuracy of synchronisation be-
tween sensors. Using commercial off-the-shelf components may compromise
quality and accuracy, because it is difficult to handle the combined data rate
from multiple sensors, the offset and rate discrepancies between independent
hardware clocks, the absence of trigger inputs or -outputs in the hardware, as
well as the different methods for timestamping the recorded data. To achieve
accurate synchronisation, we centralise the synchronisation task by recording
all trigger- or timestamp signals with a multi-channel audio interface. For
sensors that don’t have an external trigger signal, we let the computer that
captures the sensor data periodically generate timestamp signals from its se-
rial port output. These signals can also be used as a common time base to
synchronise multiple asynchronous audio interfaces. Furthermore, we show
that a consumer PC can currently capture 8-bit video data with 1024x1024
spatial- and 59.1Hz temporal resolution, from at least 14 cameras, together
with 8 channels of 24-bit audio at 96kHz. We thus improve the quality/cost
ratio of multi-sensor systems data capture systems.
bandwidth recording from multiple cameras, as well as from other sensors. In
turn, sensor fusion has increased the required accuracy of synchronisation be-
tween sensors. Using commercial off-the-shelf components may compromise
quality and accuracy, because it is difficult to handle the combined data rate
from multiple sensors, the offset and rate discrepancies between independent
hardware clocks, the absence of trigger inputs or -outputs in the hardware, as
well as the different methods for timestamping the recorded data. To achieve
accurate synchronisation, we centralise the synchronisation task by recording
all trigger- or timestamp signals with a multi-channel audio interface. For
sensors that don’t have an external trigger signal, we let the computer that
captures the sensor data periodically generate timestamp signals from its se-
rial port output. These signals can also be used as a common time base to
synchronise multiple asynchronous audio interfaces. Furthermore, we show
that a consumer PC can currently capture 8-bit video data with 1024x1024
spatial- and 59.1Hz temporal resolution, from at least 14 cameras, together
with 8 channels of 24-bit audio at 96kHz. We thus improve the quality/cost
ratio of multi-sensor systems data capture systems.
Date Issued
2010-01-01
Citation
Departmental Technical Report: 10/4, 2010, pp.1-39
Publisher
Department of Computing, Imperial College London
Start Page
1
End Page
39
Journal / Book Title
Departmental Technical Report: 10/4
Copyright Statement
© 2010 The Author(s). This report is available open access under a CC-BY-NC-ND (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Publication Status
Published
Article Number
10/4