Applied Statistics 2014 / Commenges
11th Applied Statistics 2014
International Conference
September 21 - 24, 2014
Ribno (Bled), Slovenia

Past conferences:
Applied Statistics

2013 2012 2011 2010  2009 2008 2007 2006 2005  2004
 before 2004


  Important dates

June 1
  Acceptance note
     July 1
  Reduced early
  payment fee
      July 15



Information Theory and Statistics: an overview

Daniel Commenges
Institut de Santé Publique, d'épidémiologie et de développement (ISPED), Université de Bordeaux
Institut National de la Santé de la Recherche Médicale (INSERM)

We give an overview of the role of information theory in statistics, and particularly in biostatistics. We recall the basic quantities in information theory; entropy, cross-entropy, conditional entropy, mutual information and  Kullback-Leibler risk. Then we examine the role of information theory in estimation theory. Then the basic quantities are extended to estimators, leading to criteria for estimator selection, such as Akaike criterion and its extensions. Finally we investigate the use of these concepts in Bayesian theory.


 Applied Statistics 2014                                e-mail: