empirical methods for microeconomic applications university of lugano, switzerland may 27-31, 2013
DESCRIPTION
Empirical Methods for Microeconomic Applications University of Lugano, Switzerland May 27-31, 2013. William Greene Department of Economics Stern School of Business. 2B. Heterogeneity: Latent Class and Mixed Models. Agenda for 2B. Latent Class and Finite Mixtures Random Parameters - PowerPoint PPT PresentationTRANSCRIPT
Empirical Methods for Microeconomic Applications
University of Lugano, SwitzerlandMay 27-31, 2013
William GreeneDepartment of EconomicsStern School of Business
2B. Heterogeneity: Latent Class and Mixed Models
Agenda for 2B• Latent Class and Finite
Mixtures• Random Parameters• Multilevel Models
Latent Classes• A population contains a mixture of
individuals of different types (classes)• Common form of the data generating
mechanism within the classes• Observed outcome y is governed by the
common process F(y|x,j )• Classes are distinguished by the
parameters, j.
Density? Note significant mass below zero. Not a gamma or lognormal or any other familiar density.
How Finite Mixture Models Work
ˆ
1 y - 7.05737 1 y - 3.25966F(y) =.28547 +.714533.79628 3.79628 1.81941 1.81941
Find the ‘Best’ Fitting Mixture of Two Normal Densities
1000
2 i jji=1 j=1
j j
y -μ1 LogL = log πσ σ
Maximum Likelihood Estimates Class 1 Class 2 Estimate Std. Error Estimate Std. errorμ 7.05737 .77151 3.25966 .09824σ 3.79628 .25395 1.81941 .10858π .28547 .05953 .71453 .05953
Mixing probabilities .715 and .285
Approximation
Actual Distribution
A Practical Distinction• Finite Mixture (Discrete Mixture):
• Functional form strategy• Component densities have no meaning • Mixing probabilities have no meaning• There is no question of “class membership”• The number of classes is uninteresting – enough to get a good fit
• Latent Class:• Mixture of subpopulations• Component densities are believed to be definable “groups”
(Low Users and High Users in Bago d’Uva and Jones application)• The classification problem is interesting – who is in which class?• Posterior probabilities, P(class|y,x) have meaning• Question of the number of classes has content in the context of
the analysis
The Latent Class Model
it it
(1) There are Q classes, unobservable to the analyst(2) Class specific model: f(y | ,class q) g(y , )(3) Conditional class probabilities Common multinomial logit form for prior class pr
it it qx ,x β
iq Qq 1
q q Q
obabilitiesexp(δ ) P(class=q| ) , δ = 0
exp(δ ) = log( / ).
q
δ
Log Likelihood for an LC Model
i
x x β
X ,β x βi
i
i,t i,t it i,t q
iT
i1 i2 i,T q it i,t qt 1
i
Conditional density for each observation is P(y | ,class q) f(y | , )Joint conditional density for T observations isf(y ,y ,...,y | ) f(y | , )(T may be 1. This is not
iX x βi
i
TQi1 i2 i,T q it i,t qq 1 t 1
only a 'panel data' model.)Maximize this for each class if the classes are known. They aren't. Unconditional density for individual i isf(y ,y ,...,y | ) f(y | , )Log Likelihoo
1β β x βiTN Q
Q 1 Q q it i,t qi 1 q 1 t 1
dLogL( ,..., ,δ ,...,δ ) log f(y | , )
Estimating Which Class
i
i
q
iT
i1 i2 i,T i it i,tt 1
Prob[class=q]= for T observations is
P(y ,y ,...,y | ,class q) f(y | , ) membership is the product
q
Prior class probability Joint conditional density
X x βJoint density for data and class
i
i
ii
Ti1 i2 i,T i q it i,tt 1
ii1 i2 i,T i
i1 i2 i,T i
P(y ,y ,...,y ,class q| ) f(y | , )
P( ,class q| )P(class q| y ,y ,...,y , ) P(y ,y ,...,y | )
q
i
X x βPosterior probability for class, given the data
y XX X
i
i
iQ
iq 1
Tq it i,tt 1
i i TQq it i,tq 1 t 1
P( ,class q| ) P( ,class q| )
Use Bayes Theorem to compute the f(y | , )w(q| , ) P(class j | , )
f(y | , )
i
i
qi i
q
y Xy X
posterior (conditional) probabilityx β
y X y Xx β
iq wBest guess = the class with the largest posterior probability.
‘Estimating’ βi
Qiq=1
Qiqq=1
ˆ(1) Use from the class with the largest estimated probability(2) Probabilistic - in the same spirit as the 'posterior mean'
ˆ ˆ = Posterior Prob[class=q|data]ˆˆ = w
Note:
j
i q
q
β
β β
β This estimates E[ | ], not itself.i i i iβ y ,X β
How Many Classes?(1) Q is not a 'parameter' - can't 'estimate' Q with and (2) Can't 'test' down or 'up' to Q by comparing log likelihoods. Degrees of freedom for Q+1 vs. Q classes is not well define
β
d.(3) Use AKAIKE IC; AIC = -2 logL + 2#Parameters.
The Extended Latent Class Model
it
Class probabilities relate to observable variables (usuallydemographic factors such as age and sex).(1) There are Q classes, unobservable to the analyst(2) Class specific model: f(y | ,class q) g(itx
it q
qiq qQ
qq 1
y , )(3) Conditional class probabilities given some information, ) Common multinomial logit form for prior class probabilities
exp( ) P(class=q| , ) , = exp( )
it
i
ii
i
,x βz
zδz δ δ 0zδ
Unfortunately, this argument is incorrect.
Zero Inflation?
Zero Inflation – ZIP Models• Two regimes: (Recreation site visits)
• Zero (with probability 1). (Never visit site)• Poisson with Pr(0) = exp[- ’xi]. (Number of visits,
including zero visits this season.)• Unconditional:
• Pr[0] = P(regime 0) + P(regime 1)*Pr[0|regime 1]• Pr[j | j >0] = P(regime 1)*Pr[j|regime 1]
• This is a “latent class model”
A Latent Class Hurdle NB2 Model• Analysis of ECHP panel data (1994-2001)• Two class Latent Class Model
• Typical in health economics applications• Hurdle model for physician visits
• Poisson hurdle for participation and negative binomial intensity given participation
• Contrast to a negative binomial model
LC Poisson Regression for Doctor Visits
Heckman and Singer’s RE Model• Random Effects Model• Random Constants with Discrete Distribution
it it q
q
(1) There are Q classes, unobservable to the analyst(2) Class specific model: f(y | ,class q) g(y , )(3) Conditional class probabilities Common multinomial logit form for prior clas
it itx ,x ,β
Qqq=1
qq QJ
qj 1
s probabilities to constrain all probabilities to (0,1) and ensure 1; multinomial logit form for class probabilities;
exp( ) P(class=q| ) , = 0exp( )
δ
3 Class Heckman-Singer Form
Modeling Obesity with a Latent Class Model
Mark HarrisDepartment of Economics, Curtin University
Bruce HollingsworthDepartment of Economics, Lancaster University
William GreeneStern School of Business, New York University
Pushkar MaitraDepartment of Economics, Monash University
Two Latent Classes: Approximately Half of European Individuals
An Ordered Probit ApproachA Latent Regression Model for “True BMI”
BMI* = ′x + , ~ N[0,σ2], σ2 = 1 “True BMI” = a proxy for weight is unobservedObservation Mechanism for Weight Type
WT = 0 if BMI* < 0 Normal 1 if 0 < BMI* < Overweight 2 if < BMI* Obese
Latent Class Modeling• Several ‘types’ or ‘classes. Obesity be due to
genetic reasons (the FTO gene) or lifestyle factors• Distinct sets of individuals may have differing
reactions to various policy tools and/or characteristics
• The observer does not know from the data which class an individual is in.
• Suggests a latent class approach for health outcomes(Deb and Trivedi, 2002, and Bago d’Uva, 2005)
Latent Class Application• Two class model (considering FTO gene):
• More classes make class interpretations much more difficult
• Parametric models proliferate parameters• Endogenous class membership: Two classes allow us to
correlate the equations driving class membership and observed weight outcomes via unobservables.
• Theory for more than two classes not yet developed.
Endogeneity of Class Membership
,
,
Class Membership: C* = , C = 1[C* > 0] (Probit)
BMI|Class=0,1 BMI* = , BMI group = OP[BMI*, ( )]
10Endogeneity: ~ ,
10
Bivaria
z
x w
i i
c i c i c i
i c
c i c
u
uN
te Ordered Probit (one variable is binary).
Full information maximum likelihood.
Outcome Probabilities• Class 0 dominated by normal and overweight probabilities ‘normal weight’ class• Class 1 dominated by probabilities at top end of the scale ‘non-normal weight’• Unobservables for weight class membership, negatively correlated with those
determining weight levels:
Classification (Latent Probit) Model
Inflated Responses in Self-Assessed Health
Mark HarrisDepartment of Economics, Curtin University
Bruce HollingsworthDepartment of Economics, Lancaster University
William GreeneStern School of Business, New York University
SAH vs. Objective Health MeasuresFavorable SAH categories seem artificially high.
60% of Australians are either overweight or obese (Dunstan et. al, 2001) 1 in 4 Australians has either diabetes or a condition of impaired glucose metabolism Over 50% of the population has elevated cholesterol Over 50% has at least 1 of the “deadly quartet” of health conditions (diabetes, obesity, high blood pressure, high cholestrol) Nearly 4 out of 5 Australians have 1 or more long term health conditions (National Health Survey, Australian Bureau of Statistics 2006) Australia ranked #1 in terms of obesity rates
Similar results appear to appear for other countries
A Two Class Latent Class Model
True Reporter Misreporter
• Mis-reporters choose either good or very good• The response is determined by a probit model
* m m mm x
Y=3
Y=2
Y=4
Y=3
Y=2
Y=1
Y=0
Observed Mixture of Two Classes
Pr(true,y) = Pr(true) * Pr(y | true)
Pr( ) Pr( ) Pr( | ) Pr( ) Pr( | )y true y true misreporter y misreporter
General Result
Poor Fair Good Very Good Excellent0
0.050.1
0.150.2
0.250.3
0.350.4
SamplePredictedMis-Reporting
RANDOM PARAMETER MODELS
A Recast Random Effects Modeli
i
i
it i
i
1 2 ,1
+ u + , u ~ [0, ]u
T = observations on individual iFor each period, y 1[U 0] (given u )Joint probability for T observations is
Prob( , ,... | ) ( )
For co
U
i
it i it u
i
it
Ti i i it i i
it
tt
N
y y u F y
x
x
i u u
1 , u1
1
nvenience, write u = , ~ [0,1],
log | ,... log ( ( ) )
It is not possible to maximize log | ,... because ofthe unobserved random effects embedded in
i
i i i i
TNN it i iti i t
N
i
v v N v
L v v F y v
L v v
x
.
A Computable Log Likelihood
1 1
u
log log ( , )
Maximize this function
The unobserved heterogeneity is ave
with respect to , , . ( )How to compute the integral?(1) Analytically
raged u
? N
o
ti
i
TNit i it i ii t
i u i
L F y f d
v
x
o, no formula exists.(2) Approximately, using Gauss-Hermite quadrature(3) Approximately using Monte Carlo simulation
Simulation
ii
Tit i it
Nii 1
Nii 1
Ni
2i
t
i
1
i
i1
logL log d
= log d
This ]The expected value of the function of can
equal be a
-
p
F(y , )
g(
proximas log
tedby
1 exp
dra
w
)
g
in
2
g
(
E2
)
a
[
R r
x
i
ir
i
TN RS i
r u ir
t u ir iti 1 r 1 t 1
ndom draws v from the population N[0,1] andaveraging the R functions of v . We
1logL log F(y ,( v )
maxim
)R
ize
x
Random Effects Model: Simulation----------------------------------------------------------------------Random Coefficients Probit ModelDependent variable DOCTOR (Quadrature Based)Log likelihood function -16296.68110 (-16290.72192) Restricted log likelihood -17701.08500Chi squared [ 1 d.f.] 2808.80780Simulation based on 50 Halton draws--------+-------------------------------------------------Variable| Coefficient Standard Error b/St.Er. P[|Z|>z]--------+------------------------------------------------- |Nonrandom parameters AGE| .02226*** .00081 27.365 .0000 ( .02232) EDUC| -.03285*** .00391 -8.407 .0000 (-.03307) HHNINC| .00673 .05105 .132 .8952 ( .00660) |Means for random parametersConstant| -.11873** .05950 -1.995 .0460 (-.11819) |Scale parameters for dists. of random parametersConstant| .90453*** .01128 80.180 .0000--------+-------------------------------------------------------------
Implied from these estimates is .904542/(1+.904532) = .449998.
Recast the Entire Parameter Vector
i i 1
i
1 2 ,1
ik
1
+ , , ~ [ , ( ,..., )]
Joint probability for T observations is
Prob( , ,... | ) ( )
For convenience, write u = , ~ [0,1],
log | ,.
U
.
i
i it it
i K
Ti i i it i itt
k ik ik ik k
i
k ik
t
N diag
y y u F y
v v N v
L v
xu u 0
x
,1
1
. log ( )
It is not possible to maximize log | ,... because ofthe unobserved random effects embedded in .
iTNN it i iti i t
N
i
v F y
L
x
v v
~ N[ , ]Cholesky Decomposition: = where is upper triangular
with ~ N[ , ]Convenient Refinement: ( )( ) where the diagonal elements
i
i i i
u 0LL L
u Lv v 0 IMS MS
of = 1, and is diagonal
with free positive elements. (Cholesky values)
returns the original uncorrelated casei i
M S
u MSvM I
S
M
MSSM
Modeling Parameter Heterogeneity
i,k i k
Individual heterogeneity in the means of the parameters +
E[ | , ]Heterogeneity in the variances of the parametersVar[u | data] exp( )Estimation by maximum simulated likelihood
i i i
i i i
i k
β =β Δz uu X z
hδ
Hierarchical Probit ModelUit = 1i + 2iAgeit + 3iEducit + 4iIncomeit + it.
1i=1+11 Femalei + 12 Marriedi + u1i 2i=2+21 Femalei + 22 Marriedi + u2i 3i=3+31 Femalei + 32 Marriedi + u3i 4i=4+41 Femalei + 42 Marriedi + u4i
Yit = 1[Uit > 0]
All random variables normally distributed.
Simulating Conditional Means for Individual Parameters
, ,1 1
,1 1
1
1 ˆ ˆˆ ˆ( ) (2 1)( )ˆ ( | , ) 1 ˆ ˆ (2 1)( )
1 ˆˆ =
i
i
TRi r it i r itr t
i i i TRit i r itr t
Rir irr
yRE
yR
WeightR
Lw Lw xy X
Lw x
Posterior estimates of E[parameters(i) | Data(i)]
Probit
“Individual Coefficients”
Mixed Model Estimation
• WinBUGS: • MCMC • User specifies the model – constructs the Gibbs Sampler/Metropolis Hastings
• MLWin:• Linear and some nonlinear – logit, Poisson, etc.• Uses MCMC for MLE (noninformative priors)
• SAS: Proc Mixed. • Classical• Uses primarily a kind of GLS/GMM (method of moments algorithm for loglinear models)
• Stata: Classical• Several loglinear models – GLAMM. Mixing done by quadrature.• Maximum simulated likelihood for multinomial choice (Arne Hole, user provided)
• LIMDEP/NLOGIT• Classical• Mixing done by Monte Carlo integration – maximum simulated likelihood• Numerous linear, nonlinear, loglinear models
• Ken Train’s Gauss Code• Monte Carlo integration• Mixed Logit (mixed multinomial logit) model only (but free!)
• Biogeme• Multinomial choice models• Many experimental models (developer’s hobby)
Programs differ on the models fitted, the algorithms, the paradigm, and the extensions provided to the simplest RPM, i = +wi.