On Estimation of Entropy and Mutual Information of Continuous Distribution

R. Moddemeijer

University of Twente, Department of Electrical Engineering,
P.O. Box 217, NL-7500 AE Enschede, The Netherlands,

present addres

University of Groningen, Department of Computing Science,
P.O. Box 800, NL-9700 AV Groningen, The Netherlands,
phone: +31.50.363 3940 - fax: +31.50.363 38005 - e-mail: rudy@cs.rug.nl


Mutual information is uded in a procedure to estimate time-delays between recordings of electroencephalogram (EEG) signals originating from epileptic animals or patients. We present a simple and reliable histogram-based method to estimate mutual information. The accuracies of this mutual information estimator and of a similar entropy estimator are discussed. The bias and variance calculations presented can also be applied to discrete valued systems. Finally we present some simulation results, which are compared with earlier work.


Mutual information, entropy, bias, variance, estimator, delay estimation.


Signal Processing, 1989, vol. 16, nr. 3, pp. 233-246, BibTeX

Related work

Matlab sources:

other publications