    Next: Generalizations Up: MAXIMUM LIKELIHOOD METHODS Previous: The Neerven estimator   Contents

# The estimation of the entropy

The correlation entropy can be defined by [Eckmann and Ruelle, 1985] (see Chapter 4): (5.39)

In section 5.2.4 we derived an estimator for (5.40)

using the likelihood function for a doubly censored set of data: (5.41)

To simplify the algebra (cf. appendix C), we write the likelihood function as:    (5.42)

where . The logarithm of this likelihood function is:    (5.43)

To estimate the entropy, we have to estimate for at least two embedding dimensions, since is unknown. Therefore, we now consider the case where the sample consists of distances calculated at embedding dimension and distances calculated at embedding dimension . We assume that the and distances are independent. The likelihood function for this case, , is the product of the likelihood functions and . The likelihood equation for is:    (5.44)

The likelihood equations for and are: (5.45)

and (5.46)

For the estimators of the parameters , and we find: (5.47)

and (5.48)

and a similar expression for . The estimator of the entropy is now given by: (5.49)

To obtain the variances of the dimension and entropy estimators, we have to invert the information matrix (5.50)

where    (5.51)

and (5.52)

and a similar expression for ; all other elements are zero due to our choice of parameters. For the variances of and we find: (5.53)

Using eq. (5.27) with (5.54)

we obtain an expression for the variance of the ML entropy estimator:     (5.55)

If we substitute the estimated values , and we obtain: (5.56)

and (5.57)

It is gratifying that this derivation and the one in appendix C are consistent and that both the estimators and their variances do not depend on the reference distance. They should not, because the dimension and entropy are invariant under smooth coordinate transformations [Ott et al., 1984]. In section 5.6 the validity of the expressions for the variances is supported by Monte Carlo simulations.

It can readily be verified that the covariance between and is zero if . Finally, the estimator of the entropy (eq. (5.49)) is equivalent to the one of the previous chapter (eq. (4.7), with ) if we set and if the correlation integrals are based on independent distances. Correlations between distances are discussed in section 5.6.2.    Next: Generalizations Up: MAXIMUM LIKELIHOOD METHODS Previous: The Neerven estimator   Contents
webmaster@rullf2.xs4all.nl