| MATLAB Function Reference |
entropy
Estimate the entropy of a stationary signal with independent samples
Syntax
[estimate,Nbias,sigma,descriptor] = entropy(x) [estimate,Nbias,sigma,descriptor] = entropy(x,descriptor) [estimate,Nbias,sigma,descriptor] = entropy(x,descriptor,base) [estimate,Nbias,sigma,descriptor] = entropy(x,descriptor,base,approach)
Description
Entropy estimation is a two stage process; first a
histogram is estimated and thereafter the entropy is calculated. For
the explanation of the usage of the descriptor of the histogram
see histogram .
In case of a disrete stochastic variable i in the integer subrange
lower <= i < upper the descriptor should be selected
as [lower,upper,upper-lower]. The
R(epresentation)-unbiased entropy will be estimated.
In case of a continuous stochastic variable the descriptor can be left unspecified. In this case the default descriptor of histogram will be used.
The estimate depends on the value of approach
'unbiased': a N(umber)-unbiased estimate (default),
'biased': a N(umber)-biased estimate and
'mmse': a minimum Mean Square Error estimate, obtained by
balancing bias and variance after N-bias correction.
base of the logarithm determines the unit of
measurement. Default base e (nats) is used, alternative choises are 2 (bit)
and 10 (Hartley).
As a result the function returns the estimate, the N-bias
(Nbias) of the estimate, the estimated standard error sigma and the used descriptor.
Example
Estimate the entropy of a discrete stochastic variable with probability 0.25
on an observed 0 and probability 0.75 on an observed 1. The entropy is
0.5623 nat.
>> signal(1:250)=0;
>> signal(251:1000)=1;
>> [estimate,nbias,sigma,descriptor] =entropy(signal,[0,2,2],'u')
estimate = 0.5628
nbias = 0
sigma = 0.0151
descriptor = 0 2 2
Estimate the entropy of a Gaussian distributed continuous variable with zero mean and unit variance. The entropy is 1.4189 nat.
>> signal=normrnd(0,1,1,1000);
>> [estimate,nbias,sigma,descriptor] =entropy(signal,[-3,3,30])
estimate = 1.3643
nbias = 0
sigma = 0.0208
descriptor = -3 3 30
See Also
Literature
Moddemeijer, R. On Estimation of Entropy and Mutual Information of Continuous Distributions, Signal Processing, 1989, vol. 16, nr. 3, pp. 233-246, abstract , BibTeX ,
For the principle of Minimum Mean Square Error estimation see:
Moddemeijer, R. An efficient algorithm for selecting optimal configurations of AR-coefficients, Twentieth Symp. on Information Theory in the Benelux, May 27-28, 1999, Haasrode (B), pp 189-196, eds. A. Barbé et. al., Werkgemeenschap Informatie- en Communicatietheorie, Enschede (NL), and IEEE Benelux Chapter on Information Theory, ISBN: 90-71048-14-4, abstract , BibTeX ,
Source code
| MATLAB Function Reference |