R: Dirichlet Prior Bayesian Estimators of Entropy, Mutual...
entropy.Dirichlet
R Documentation
Dirichlet Prior Bayesian Estimators of Entropy, Mutual Information
and Other Related Quantities
Description
freqs.Dirichlet computes the Bayesian estimates
of the bin frequencies using the Dirichlet-multinomial
pseudocount model.
entropy.Dirichlet estimates the Shannon entropy H of the random variable Y
from the corresponding observed counts y by plug-in of Bayesian estimates
of the bin frequencies using the Dirichlet-multinomial
pseudocount model.
KL.Dirichlet computes a Bayesian estimate of the Kullback-Leibler (KL) divergence
from counts y1 and y2.
chi2.Dirichlet computes a Bayesian version of the chi-squared statistic
from counts y1 and y2.
mi.Dirichlet computes a Bayesian estimate of mutual information of two random variables.
chi2indep.Dirichlet computes a Bayesian version of the chi-squared statistic of
independence from a table of counts y2d.
Usage
freqs.Dirichlet(y, a)
entropy.Dirichlet(y, a, unit=c("log", "log2", "log10"))
KL.Dirichlet(y1, y2, a1, a2, unit=c("log", "log2", "log10"))
chi2.Dirichlet(y1, y2, a1, a2, unit=c("log", "log2", "log10"))
mi.Dirichlet(y2d, a, unit=c("log", "log2", "log10"))
chi2indep.Dirichlet(y2d, a, unit=c("log", "log2", "log10"))
Arguments
y
vector of counts.
y1
vector of counts.
y2
vector of counts.
y2d
matrix of counts.
a
pseudocount per bin.
a1
pseudocount per bin for first random variable.
a2
pseudocount per bin for second random variable.
unit
the unit in which entropy is measured.
The default is "nats" (natural units). For
computing entropy in "bits" set unit="log2".
Details
The Dirichlet-multinomial pseudocount entropy estimator
is a Bayesian plug-in estimator:
in the definition of the Shannon entropy the
bin probabilities are replaced by the respective Bayesian estimates
of the frequencies, using a model with a Dirichlet prior and a multinomial likelihood.
The parameter a is a parameter of the Dirichlet prior, and in effect
specifies the pseudocount per bin. Popular choices of a are:
a=0:maximum likelihood estimator (see entropy.empirical)