Entropy Inference and the James-Stein Estimator, with Application to Nonlinear Gene Association Networks
File(s)p1469-hausser.pdf (145.1 KB)
Published version
Author(s)
Hausser, J
Strimmer, K
Type
Journal Article
Abstract
We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically as well as computationally. Despite its simplicity, we show that it outperforms eight other entropy estimation procedures across a diverse range of sampling scenarios and data-generating models, even in cases of severe undersampling. We illustrate the approach by analyzing E. coli gene expression data and computing an entropy-based gene-association network from gene expression data. A computer program is available that implements the proposed shrinkage estimator.
Date Issued
2009-07-30
Date Acceptance
2009-07-01
Citation
Journal of Machine Learning Research, 2009, 10, pp.1469-1484
ISSN
1532-4435
Publisher
Journal of Machine Learning Research
Start Page
1469
End Page
1484
Journal / Book Title
Journal of Machine Learning Research
Volume
10
Copyright Statement
© 2009 Jean Hausser and Korbinian Strimmer
Identifier
http://arxiv.org/abs/0811.3579v3
Subjects
stat.ML
Artificial Intelligence & Image Processing
08 Information And Computing Sciences
17 Psychology And Cognitive Sciences
Publication Status
Published