| MATLAB Function Reference |
information
Estimate the mutual information of two stationary signal with independent pairs of samples
Syntax
[estimate,Nbias,sigma,descriptor] = information(x,y) [estimate,Nbias,sigma,descriptor] = information(x,y,descriptor) [estimate,Nbias,sigma,descriptor] = information(x,y,descriptor,base) [estimate,Nbias,sigma,descriptor] = information(x,y,descriptor,base,approach)
Description
Mutual information estimation is like entropy estimation a two stage
process; first a two demensional histogram2
is estimated and thereafter the mutual information is calculated. For the
explanation of the usage of the descriptor of the histogram see
histogram2 .
In case of a disrete stochastic variable i and j in the integer subranges
lowerx <= i < upperx and lowery <= j < uppery the descriptor should be selected
as [lowerx,upperx,upperx-lowerx;lowery,uppery,uppery-lowery]. The
R(epresentation)-unbiased entropy will be estimated.
In case of a continuous stochastic variable the descriptor can be left unspecified. In this case the default descriptor of histogram2 will be used.
The estimate depends on the value of approach
'unbiased': a N(umber)-unbiased estimate (default),
'biased': a N(umber)-biased estimate and
'mmse': a minimum Mean Square Error estimate, obtained by
balancing bias and variance after N-bias correction.
base of the logarithm determines the unit of
measurement. Default base e (nats) is used, alternative choises are 2 (bit)
and 10 (Hartley).
As a result the function returns the estimate, the N-bias
(Nbias) of the estimate, the estimated standard error sigma and the used descriptor.
Note: due to samples outside the histogram, which are excuded from the
estimate, the relation entropy(x)+entropy(y)-estropy2(x,y) ==
information(x,y) is only approximately valid.
Example
The mutual information of binormal distributed pairs of samples with
correlation coefficient rho is 0.5 equals 0.1438 nat.
Estimate this mutual information:
>> rho=0.5;
>> x=normrnd(0,1,1,1000);
>> n=normrnd(0,1,1,1000);
>> y=rho*x+sqrt(1-rho^2)*n;
>> [estimate,nbias,sigma,descriptor] =information(x,y,[-3,3,12;-3,3,12])
estimate = 0.1209
nbias = 0
sigma = 0.0173
descriptor = -3 3 12
-3 3 12
See Also
See Also
Literature
Moddemeijer, R. On Estimation of Entropy and Mutual Information of Continuous Distributions, Signal Processing, 1989, vol. 16, nr. 3, pp. 233-246, abstract , BibTeX ,
For the principle of Minimum Mean Square Error estimation see:
Moddemeijer, R. An efficient algorithm for selecting optimal configurations of AR-coefficients, Twentieth Symp. on Information Theory in the Benelux, May 27-28, 1999, Haasrode (B), pp 189-196, eds. A. Barbé et. al., Werkgemeenschap Informatie- en Communicatietheorie, Enschede (NL), and IEEE Benelux Chapter on Information Theory, ISBN: 90-71048-14-4, abstract , BibTeX ,
Source code
| MATLAB Function Reference |