The variance of the mutual information estimator

R. Moddemeijer

University of Groningen, Department of Computing Science,
P.O. Box 800, NL-9700 AV Groningen, The Netherlands,
phone: +31.50.363 3940 - fax: +31.50.363 38005 - e-mail: rudy@cs.rug.nl

Abstract

In the case of two signals with independent pairs of observations a statistic to estimate the variance of the mutual information estimator has been derived earlier. We present such a statistic for dependent pairs. To derive this statistic it is necessary to avail of a reliable statistic to estimate the variance of the sample mean in case of dependent observations. We derive and discuss this statistic and a statistic to estimate the variance of the mutual information estimator. These statistics are validated by simulations.

Keywords

sample mean, variance, statistic, mutual information, entropy, likelihood, dependent samples.

Full paper


Published

University of Groningen, Department of Computing Science, Groningen, Computing Science Report: CS-R9701, BibTeX
other publications