LEAST SQUARES AND ENTROPY AS PENALTY FUNCTIONS

Mathematical measures of entropy as defined by Shannon (1948) and Kullback and Leibler (1951) are currently in vogue in the field of econometrics, primarily due to the comprehensive work by Golan, Judge, and Miller (1996). In this paper, an alternative interpretation of the entropy measure as a penalty function over deviations is presented. Using this interpretation, a number of parallels are drawn with least squares estimators, and it is demonstrated that, with a minor modification of the traditional least squares estimator, both approaches may be applied to the general linear model. The advantages and disadvantages of each approach are discussed, and a philosophical approach to the selection of estimation technique is suggested.


Issue Date:
1998
Publication Type:
Working or Discussion Paper
PURL Identifier:
http://purl.umn.edu/28625
Total Pages:
13
Series Statement:
Staff Paper 98-16




 Record created 2017-04-01, last modified 2017-08-22

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)