Ambient and wearable sensor fusion for privacy respectful pervasive monitoring environments (Thesis)

Abstract

Computational devices are becoming smaller and are transforming the way we conduct our lives. This opens up a wealth of new opportunities for applications that would previously only have been considered possible in the context of science fiction. One of the most exciting areas of development is the use of miniaturised devices to form Body Sensor Networks (BSNs). This has wide reaching applications in the field of healthcare - where long term trend analysis can be used to assess the efficacy of treatments or to chart the progress of recovery; and sport - where a greater understanding of human motion can be used to provide rapid, accurate feedback to athletes during training to maximise performance. In order to provide an accurate picture of human motion, many are beginning to realise the efficacy of sensor fusion, where multiple sensors or sensing modalities are used to provide an advantage over a single sensor. In this thesis, we investigate how sensor fusion can be used in a very specific circumstance - a privacy respectful home monitoring environment. That is, an environment in which sensitive information, such as appearance, should not be extracted, stored, transmitted or used. This provides us with a challenging research area with direct applicability to a real world scenario. This thesis contributes to existing research in BSNs in several ways. Firstly, we augment an existing Receiver Operating Characteristic (ROC) based technique known as the Bayesian Framework for Feature Selection (BFFS), in order to choose optimal feature sets from disparate sensors which when used to classify activity, have maximal overlap. This provides the groundwork for a probabilistic framework to temporally match data streams without the requirement for data which can be used to identify users. The selection and use of simple classifiers also ensures that, in a home healthcare scenario, classification and matching could occur on node and in real-time. We validate this approach using both expected area under the ROC curve, ROC response from a cross validated naïve Bayesian classifier and the true level of overlap when a naïve Bayesian classifier is trained in a leave one out fashion. With a privacy respectful method for the matching of data from disparate sources, we proceed to investigate the recognition of activities, such as those which may be representative of Activities of Daily Living (ADLs). To this end, we discuss the generation of privacy respectful metrics of human activity and develop a wrapper-based feature selection method that uses multiple, simple classifiers to generalise the quality of selected features without preference to any given classifier. We term this Ensemble Feature Selection (EFS). We validate EFS against the existing BFFS framework. Using EFS and BFFS selected features, we demonstrate that sensor fusion is advantageous for monitoring ADLs including eating, sleeping, walking, reading, sitting and lounging. The final contribution of this thesis is the demonstration of a motion detection system that provides detailed information regarding subject pose and motion through the use of calibrated cameras. A technique is utilised which extracts convex hulls probabilistically. This defers the decision of foreground membership until back-propagation to all cameras in the environment. Segmentation is performed through the combination of locally held beliefs regarding the background distribution. A canonical descriptor is then extracted through cylindrical re-projection of the convex hull and features derived. Such features are simple in nature and privacy respectful, yet contain a high level of detail regarding subject pose - properties that are ideal for the monitoring of subjects within a home environment. A detailed framework for recognition of complex motions using spatial and temporal Hidden Markov Models (HMMs) is provided and we demonstrate how to augment the system with a wearable sensor for improved recognition accuracy.