Plug-In Estimator of the Kullback-Leibler divergence and of the Chi-Squared Statistic

Description

KL.plugin computes the Kullback-Leiber (KL) divergence from random variable X_1 to X_2. The corresponding probability mass functions are given by freqs1 and freqs2, and the expectation is computed over freqs1.

chi2.plugin computes the chi-squared statistic between an observed X_1 and an expected X_2, where freqs1 and freqs2 are the corresponding probability mass functions.

Usage

KL.plugin(freqs1, freqs2, unit=c("log", "log2", "log10"))
chi2.plugin(freqs1, freqs2, unit=c("log", "log2", "log10"))

Arguments

Details

Kullback-Leibler divergence between the from random variable X_1 to X_2 is given as E_{X_1} log (f(x_1)/f(x_2)) .

The chi-squared statistic is given ∑ (f(x_1)-f(x_2))^2/f(x_2) . It can also be seen as a second-order accurate approximation of twice the KL divergence.

Note that both the KL divergence and the chi-squared statistic are not symmetric in X_1 and X_2.

Value

KL.plugin returns the KL divergence.

chi2.plugin returns the chi-squared statistic.

Author(s)

Korbinian Strimmer (http://strimmerlab.org).

See Also

KL.Dirichlet, KL.shrink, KL.empirical, mi.plugin, discretize2d.

Examples

# load entropy library 
library("entropy")

# probabilities for two random variables
freqs1 = c(1/5, 1/5, 3/5)
freqs2 = c(1/10, 4/10, 1/2) 

# KL divergence from X1 to X2
KL.plugin(freqs1, freqs2)

# and corresponding (half) chi-squared statistic
0.5*chi2.plugin(freqs1, freqs2)

Results