A Common-Factor Approach for Multivariate Data Cleaning with an Application to Mars Phoenix Mission Data

File Description SizeFormat 
Fang etal ArXiv 2015 CS-AI 1510-01291.pdfPublished version2.14 MBAdobe PDFDownload
Title: A Common-Factor Approach for Multivariate Data Cleaning with an Application to Mars Phoenix Mission Data
Author(s): Fang, D
Oberlin, E
Ding, W
Kounaves, SP
Item Type: Working Paper
Abstract: Data quality is fundamentally important to ensure the reliability of data for stakeholders to make decisions. In real world applications, such as scientific exploration of extreme environments, it is unrealistic to require raw data collected to be perfect. As data miners, when it is infeasible to physically know the why and the how in order to clean up the data, we propose to seek the intrinsic structure of the signal to identify the common factors of multivariate data. Using our new data driven learning method, the common-factor data cleaning approach, we address an interdisciplinary challenge on multivariate data cleaning when complex external impacts appear to interfere with multiple data measurements. Existing data analyses typically process one signal measurement at a time without considering the associations among all signals. We analyze all signal measurements simultaneously to find the hidden common factors that drive all measurements to vary together, but not as a result of the true data measurements. We use common factors to reduce the variations in the data without changing the base mean level of the data to avoid altering the physical meaning.
Publication Date: 31-Dec-2015
URI: http://hdl.handle.net/10044/1/56736
Copyright Statement: © The Authors
Keywords: cs.AI
cs.AI
Appears in Collections:Faculty of Engineering
Earth Science and Engineering



Items in Spiral are protected by copyright, with all rights reserved, unless otherwise indicated.

Creative Commons