Estimation of entropy and mutual information of continuous distributions

R. Moddemeijer

University of Twente, Department of Electrical Engineering,
P.O. Box 217, NL-7500 AE Enschede, The Netherlands,

present addres

University of Groningen, Department of Computing Science,
P.O. Box 800, NL-9700 AV Groningen, The Netherlands,
phone: +31.50.363 3940 - fax: +31.50.363 38005 - e-mail:


For time-delay estimation with a maximum average amount of mutual information (AAMI) method, applied to EEG's of epileptic animals or patients, it is necessary to have a simple and reliable method for the estimation of mutual information. We present a histogram based method for estimating a two-dimensional continuous probability density, from which we determine estimatesof the entropy and the mutual information. For this method we derived expressions for bias-correction and variance estimation. We present some simulation results using a binormal distribution and hope to present the first results of the method applied to real EEG's.


Sixth Symposium on Information Theory in the Benelux, May 23-24, 1985, Mierlo, The Netherlands, pp. 27-34, Ed. Vinck, A.J., Werkgemeenschap Informatie- en Communicatietheorie, Enschede, ISBN 90-71048-02-0, BibTeX
other publications