The Method of Least Squares Hervé Abdi 1 1 Introduction The le ast square me thod s( L SM) is pr obably the mo st popula r te ch- nique in statistics. This is due to several factors. First, most com- mon estimator s can be casted within this framework. For exam- pl e, the me an of a distri bu tion is th e va lue that mini mi zes the sum of squared deviations of the scores. Second, using squares makes L SM mathemat ically very tractable because the Pythagor ean theo- rem indicates that, when the error is independent of an estimated quanti ty , one can ad d th e squarederror an d th e squaredestimated qua nti ty . Thi rd , the mathe matic al too ls and alg ori thms inv olv ed in L SM (derivatives , eigendecomposition, singular value decomposi- tion) have been well studied for a relatively long time. LS M is one of the oldest techniques of modern statistics, and even though ancestors ofL SM can be traced up to Greek mathe- mati cs, th e first mod ern precur sor is pro babl y Gal ileo (s ee Harpe r , 1974, for a history and pre-history ofLS M). The modern approach was first exposed in 1805 by the French mathematician Legendre in a now classic memoir, but this method is somewhat older be- cause it turned out that, after the publication of Legendre’s mem- oir , Gauss (the famous German mathematician) contested Legen- 1 In: Neil Salkind (Ed.) (2007). Encyclopedia of Measurement and Statistics. Thousand Oaks (CA): Sage. Address corr espondence to: Hervé Abdi Program in Cognition and Neurosciences, MS: Gr.4.1, The University of Texas at Dallas, Richardson, TX 75083–0688, USAE-mail:herve@ utdall as.edu http: //www. utd.e du/∼herve 1
