regression with limited dependent variables - alumni · regression with limited dependent variables...

62
Regression with limited dependent variables Professor Bernard Fingleton

Upload: vanhanh

Post on 26-Apr-2019

229 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Regression with limited dependent variables

• Professor Bernard Fingleton

Page 2: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Regression with limited dependent variables

• Whether a mortgage application is accepted or denied

• Decision to go on to higher education• Whether or not foreign aid is given to a

country• Whether a job application is successful• Whether or not a person is unemployed• Whether a company expands or contracts

Page 3: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Regression with limited dependent variables

• In each case, the outcome is binary• We can treat the variable as a success (Y

= 1) or failure (Y = 0)• We are interested in explaining the

variation across people, countries or companies etc in the probability of success, p = prob( Y =1)

• Naturally we think of a regression model in which Y is the dependent variable

Page 4: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Regression with limited dependent variables

• But the dependent variable Y and hence the errors are not what is assumed in ‘normal’ regression– Continuous range– Constant variance (homoscedastic)

• With individual data, the Y values are 1(success) and 0(failure)– the observed data for N individuals are discrete

values 0,1,1,0,1,0, etc……not a continuum– The variance is not constant (heteroscedastic)

Page 5: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Bernoulli distributionprobability of a success ( 1) is probability of failure ( 0) is 1-

( )var( ) (1 )as varies for i= 1,...,N individualsthen both mean and variance vary

( )var( ) (1 )regressi

i

i i

i i i

Y pY p q

E Y pY p p

p

E Y pY p p

== =

== −

== −

1

on explains variation in ( ) as a function ofsome explanatory variables

( ) ( ,..., )but the variance is not constant as ( ) changeswhereas in OLS regression, we assume only the meanvarie

i i

i i Ki

i

E Y p

E Y f X XE Y

=

=

s as varies, and the variance remains constantX

Page 6: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

The linear probability model

0 1 1

1 0 1 1

1

1 2

this is a linear regression model....

Pr( 1 ,...., ) .... is the change in the probability that = 1 associated

with a unit change in , holding constant ..

i i K Ki i

i i Ki i K Ki

Y b b X b X eY X X b b X b X

b YX X

= + + +

= = + +

.. , etcThis can be estimated by OLS but Note that since var( ) is not constant, we need to allowfor heteroscedasticity in , tests and confidence intervals

K

i

X

Yt F

Page 7: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

The linear probability model

• 1996 Presidential Election• 3,110 US Counties• binary Y with 0=Dole, 1=Clinton

Page 8: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Ordinary Least-squares Estimates R-squared = 0.0013 Rbar-squared = 0.0010 sigma^2 = 0.2494 Durbin-Watson = 0.0034 Nobs, Nvars = 3110, 2 *************************************************************** Variable Coefficient t-statistic t-probability Constant 0.478917 21.962788 0.000000 prop-gradprof 0.751897 2.046930 0.040749 prop-gradprof = pop with grad/professional degrees as a proportion of educated (at least high school education)

The linear probability model

Page 9: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35-0.5

0

0.5

1

1.5

The linear probability model

prop-gradprof = pop with grad/professional degrees as a proportion of educated (at least high school education)

Clinton

Dole

Page 10: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

0

0.2

0.4

0.6

0.8

1

0 0.05 0.1 0.15 0.2 0.25 0.3

Dole

_Clin

ton_1

prop_gradprof

Dole_Clinton_1 versus prop_gradprof (with least squares fit)

Y = 0.479 + 0.752X

The linear probability model

Page 11: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

The linear probability model

• Limitations• The predicted probability exceeds 1 as X

becomes largeˆ 0.479 0.752

ˆif 0.693 then 1ˆ0 gives 0.479

if < 0 possible, thenˆ< -0.637 gives 0

Y X

X Y

X YX

X Y

= +

> >

= =

<

Page 12: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Solving the problem• We adopt a nonlinear specification that forces

the dependent proportion to always lie within the range 0 to 1

• We use cumulative probability functions (cdfs) because they produce probabilities in the 0 1 range

• Probit– Uses the standard normal cdf

• Logit– Uses the logistic cdf

Page 13: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Probit regression

( ) area to left of z in standard normal distribution( 1.96) 0.025(0) 0.5(1) 0.84(3.0) 0.999

we can put any value for from - to + , and the outcomeis 0 = ( ) 1

z

zp z

Φ =Φ − =Φ =Φ =Φ =

∞ ∞< Φ <

Page 14: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success
Page 15: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

1 2 0 1 1 2 2

0 1 2

1 2

0 1 1 2 2

1 2

Pr( 1 , ) ( ). .

1.6, 2, 0.50.4, 1

1.6 2 0.4 0.5 1 0.3Pr( 1 , ) ( 0.3) 0.38

Y X X b b X b Xe gb b bX Xz b b X b X x x

Y X X

= = Φ + +

= − = =

= == + + = − + + = −

= = Φ − =

Probit regression

Page 16: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Probit regression

Model 9: Probit estimates using the 3110 observations 1-3110Dependent variable: Dole_Clinton_1 VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -2.11372 0.215033 -9.830 prop_gradprof 9.35232 1.32143 7.077 3.72660 log_urban 15.9631 5.64690 2.827 6.36078 prop_highs 3.07148 0.310815 9.882 1.22389

Page 17: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Probit regression

0

0.2

0.4

0.6

0.8

1

0 0.05 0.1 0.15 0.2 0.25 0.3

Dole

_Clin

ton_1

prop_gradprof

Actual and fitted Dole_Clinton_1 versus prop_gradprof

fittedactual

Page 18: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Probit regression

• Interpretation• The slope of the line is not constant• As the proportion of graduate professionals

goes from 0.1 to 0.3, the probability of Y=1 (Clinton) goes from 0.5 to 0.9

• As the proportion of graduate professionals goes from 0.3 to 0.5 the probability of Y=1 (Clinton) goes from 0.9 to 0.99

Page 19: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Probit regression

• Estimation• The method is maximum likelihood (ML)• The likelihood is the joint probability given

specific parameter values• Maximum likelihood estimates are those

parameter values that maximise the probability of drawing the data that are actually observed

Page 20: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

1 0 1 1

1 0 1 1

Pr( 1) conditional on ,..., is ( ... )Pr( 0) conditional on ,..., is 1- 1 ( ... )

is the value of observed for individual

for 'th individual, Pr(

i i Ki i i K Ki

i i Ki i i K Ki

i i

Y X X p b b X b XY X X p b b X b X

y Y i

i Y

= = Φ + += = −Φ + +

[ ] [ ]

{ }

1

1

10 1 1 0 1 1

0 1 1

) is (1 )

for 1,.., , joint likelihood is Pr( ) (1 )

Pr( ) ( ... ) 1 ( ... )

log likelihood is

ln ln ( ...

i i

i i

i i

y yi i i i

y yi i i ii i

y yi i i K Ki i K Kii i

i i K Ki

y p p

i n L Y y p p

L Y y b b X b X b b X b X

L y b b X b X

= −

= = Π = = Π −

= Π = = Π Φ + + −Φ + +

= Φ + + + { }0 1 1

0 1

(1 )ln 1 ( ...

we obtain the values of , ,..., that give the maximum value of ln

i i K Kii

K

y b b X b X

b b b L

− −Φ + +⎡ ⎤⎣ ⎦∑

Probit regression

Page 21: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

success X1 100 20 31 91 51 80 40 51 111 120 30 41 120 81 140 31 111 90 40 61 71 90 30 1

Hypothetical binary data

Page 22: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Iteration 0: log likelihood = -13.0598448595 Iteration 1: log likelihood = -6.50161713610 Iteration 2: log likelihood = -5.50794602456 Iteration 3: log likelihood = -5.29067548323 Iteration 4: log likelihood = -5.26889753239 Iteration 5: log likelihood = -5.26836878709 Iteration 6: log likelihood = -5.26836576121 Iteration 7: log likelihood = -5.26836575008 Convergence achieved after 8 iterations Model 3: Probit estimates using the 24 observations 1-24 Dependent variable: success VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -4.00438 1.45771 -2.747 X 0.612845 0.218037 2.811 0.241462

Page 23: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Model 3: Probit estimates using the 24 observations 1-24 Dependent variable: success VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -4.00438 1.45771 -2.747 X 0.612845 0.218037 2.811 0.241462

If X = 6, probit = ( 4.00438 0.612845*6) ( -0.32731) 0.45Φ − + =Φ =

Page 24: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12 14

succ

ess

X

Actual and fitted success versus X

fittedactual

Page 25: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Logit regression

• Based on logistic cdf• This looks very much like the cdf for the

normal distribution• Similar results • The use of the logit is often a matter of

convenience, it was easier to calculate before the advent of fast computers

Page 26: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Logistic function

-3 -2 -1 0 1 2 3 4

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

logit

EPR

O1

z

f(z)

Page 27: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

1

1

Prob success = 11

1 1Prob fail = 1- 1 11 1 1 1Prob success 1odds ratio =

Prob fail 1 1log odds ratio =

Zz

Z

Z Z Zz

Z Z Z Z

Z ZZ

Z

e ee

e e e ee e e e

e e ee

z

−−

−−

⎡ ⎤= +⎣ ⎦++ ⎡ ⎤= − = = − +⎣ ⎦+ + + +

+= =+

0 1 1 2 2 ..... k kz b b X b X b X= + + + +

Page 28: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Logistic function

pp plotted against plotted against XX is a straight line with is a straight line with pp <0 and >1 possible<0 and >1 possible

0 1ip b b X= +

0 1

0 1

0 1

exp( )1 exp( )

plotted against gives s-shaped logistic curveso 1 and 0 impossibleequivalently

ln1

this is the equation of a straight line, so

ln plotte1

i

i

i i

i

i

i

i

b b Xpb b X

p Xp p

p b bp

pp

+=

+ +

> <

⎧ ⎫= +⎨ ⎬−⎩ ⎭

⎧ ⎫⎨ ⎬−⎩ ⎭

d against is linearX

X

Page 29: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Estimation - logit

0 1

0 1

0 1

is fixed data, so we choose , , hence exp( )

1 exp( )so that the likelihood is maximized

i

i

X b b pb b Xp

b b X+

=+ +

Page 30: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Logit regression0 1 1

11

11

...

Pr( 1) conditional on ,..., is [1 exp( )]

Pr( 0) conditional on ,..., is 1- 1 [1 exp( )] is the value of observed for individual

for 'th individual,

i K Ki

i i Ki i

i i Ki i

i i

z b b X b X

Y X X p z

Y X X p zy Y i

i

= + +

= = + −

= = − + −

{ }

1

1

11 1

1

Pr( ) is (1 )

for 1,.., , joint likelihood is Pr( ) (1 )

Pr( ) [1 exp( )] 1 [1 exp( )]

log likelihood is

ln ln [1 exp( )] (1 )l

i i

i i

i i

y yi i i i

y yi i i ii i

y y

i ii i

i i

Y y p p

i n L Y y p p

L Y y z z

L y z y

−− −

= −

= = Π = = Π −

⎡ ⎤ ⎡ ⎤= Π = = Π + − − + −⎣ ⎦ ⎣ ⎦

= + − + − { }1

0 1

n 1 [1 exp( ( )]

we obtain the values of , ,..., that give the maximum value of lni

K

z

b b b L

−⎡ ⎤− + −⎣ ⎦∑

Page 31: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Estimation

• maximum likelihood estimates of the parameters using an iterative algorithm

Page 32: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

EstimationIteration 0: log likelihood = -13.8269570846 Iteration 1: log likelihood = -6.97202524093 Iteration 2: log likelihood = -5.69432863365 Iteration 3: log likelihood = -5.43182376684 Iteration 4: log likelihood = -5.41189406278 Iteration 5: log likelihood = -5.41172246346 Iteration 6: log likelihood = -5.41172244817 Convergence achieved after 7 iterations Model 1: Logit estimates using the 24 observations 1-24 Dependent variable: success VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -6.87842 2.74193 -2.509 X 1.05217 0.408089 2.578 0.258390

Page 33: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -6.87842 2.74193 -2.509 X 1.05217 0.408089 2.578 0.258390 If X = 6, logit = -6.87842 + 1.05217*6 =-0.5654 = ln(p/(1-p) P = exp(-0.5654)/{1+exp(-0.5654)} = 1/(1+exp(0.5654)) = 0.362299

Page 34: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

0

0.2

0.4

0.6

0.8

1

2 4 6 8 10 12 14

succ

ess

X

Actual and fitted success versus X

fittedactual

Page 35: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Modelling proportions and percentages

Page 36: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

The linear probability modelconsider the following individual data for and

0,0,1,0,0,1,0,1,1,11,1,2, 2,3,3,4, 4,5,5

constant 1,1,1,1,1,1,1,1,1,1ˆ 0.1 0.2 is the OLS estimateNotice that the values for individuals 1 and 2

Y XYX

Y XX

==

=

= − +are identical,

likewise 3 and 4 and so onIf we group the identical data, we have a set of proportions

= 0/2, 1/2, 1/2, 1/2, 1 = 0, 0.5, 0.5, 0.5, 1X = 1,2,3,4,5ˆ 0.1 0.2 is the OLS estimate

p

p X= − +

Page 37: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

The linear probability model• When n individuals are identical in terms of the

variables explaining their success/failure• Then we can group them together and explain

the proportion of ‘successes’ in n trials– This data format is often important with say

developing country data, where we know the proportion, or % of the population in each country with some attribute, such as the % of the population with no schooling

– And we wish to explain the cross country variations in the %s by variables such as GDP per capita or investment in education, etc

Page 38: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Regression with limited dependent variables

• With individual data, the values are 1(success) and 0(failure) and p is the probability that Y = 1– the observed data for N individuals are discrete

values 0,1,1,0,1,0, etc……not a continuum• With grouped individuals the proportion p is

equal to the number of successes Y in n trials (individuals)

• So the range of Y is from 0 to n• The possible Y values are discrete, 0,1,2,…,n,

and confined to the range 0 to n.• The proportions p are confined to the range 0 to

1

Page 39: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Modelling proportions

Proportion (Proportion (Y/nY/n) Continuous response) Continuous response

5/10 = 0.5 11.325/10 = 0.5 11.321/3 = 0.333 17.881/3 = 0.333 17.886/9 = 0.666 3.326/9 = 0.666 3.321/10 = 0.1 11.761/10 = 0.1 11.767/20 = 0.35 1.117/20 = 0.35 1.111/2 = 0.5 0.03 1/2 = 0.5 0.03

Page 40: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Binomial distribution

the moments of the number of successes trials, each independent, 1,..., is the probability of a success in each trial

( )var( ) (1 )the variance is not constant, but depends on

i

i

i

i i i

i i i i

Yn i NpE Y n p

Y n p p

=

== −

and ~ ( , )

i i

i i i

n pY B n p

Page 41: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Data

q0.1692110.4718630.0443430.2745890.277872

starts(n) expanded(Y) propn =Y/n13 8 0.6153834 34 1.0000010 0 0.0000015 9 0.6000016 14 0.87500

Cleveland,DurhamCumbria

NorthhumberlandHumberside

N Yorks

Region Region Output growth survey of startup firms Output growth survey of startup firms

Page 42: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

-0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.0

0.5

1.0

gvagr

prop

n =e

/s

Y = 0.296428 + 1.41711XR-Sq = 48.9 %

Regression Plot

The linear probability model

Page 43: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

OLS regression with proportions

y = 0.296 + 1.42 x

Predictor Coef StDev T PConstant 0.29643 0.05366 5.52 0.000x 1.4171 0.2559 5.54 0.000

S = 0.2576 R-Sq = 48.9% R-Sq(adj) = 47.3%

0.48310-0.005761.078920.586340.25346

Fitted values = Fitted values = y = 0.296 + 1.42 x

Negative proportionNegative proportionProportion > 1Proportion > 1

Page 44: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Grouped Data

q0.1692110.4718630.0443430.2745890.277872

starts(n) expanded(Y) propn =Y/n13 8 0.6153834 34 1.0000010 0 0.0000015 9 0.6000016 14 0.87500

Cleveland,DurhamCumbria

NorthhumberlandHumberside

N Yorks

Region Region Output growth survey of startup firms Output growth survey of startup firms

Page 45: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Proportions and counts

0 1

0 1

ln( / (1 )ˆ ˆˆ ˆln( / (1 )

( )ˆ ˆ

i i

i i

i i i

i i i

p p b b X

p p b b XE Y n p

Y n p

− = +

− = +=

=

iY

= size of sample i= size of sample i

= estimated expected number of = estimated expected number of ‘‘successessuccesses’’ in sample iin sample i

in

Page 46: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Binomial distribution

!Pr ( ) (1 )!( )!

y n ynob Y y p py n y

−= = −−

For region iFor region i

n = number of individuals= number of individuals

p = probability of a = probability of a ‘‘successsuccess’’

Y = number of = number of ‘‘successessuccesses’’ in n individualsin n individuals

Page 47: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Binomial distribution

!Pr ( ) (1 )!( )!

y n ynob Y y p py n y

−= = −−

For region iFor region i

ExampleExample PP =0.5, =0.5, nn = 10= 10

5 510!Pr ( 5) 0.5 (0.5) 0.24615!(10 5)!

( ) 5var( ) (1 ) 2.5

ob Y

E Y npY np p

= = =−

= == − =

Page 48: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

1 2 3 4 5 6 7 8 9 10

0

100

200

300

400

500

C25

Freq

uenc

y

Y is B(10,0.5) E(Y)=np = 5 var(Y)=np(1-p)=2.5

1 2 3 4 5 6 7 8 9 10

0

100

200

300

400

500

C27

Freq

uenc

y

Y is B(10,0.9) E(Y)=np = 9 var(Y)=np(1-p)=0.9

Page 49: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

1 1 1

1 1 2 2

1 2

11 1 1

1 1 1

Assume the data observed are 5 successes from 10 trials and 9 successes from 10 trials

what is the likelihood of these data given 0.5, 0.9?!Pr ob( 5) (1 )

!( )!

Pr ob

y n y

Y y Y yp p

nY p py n y

= = = == =

= = −−

2 2 2

5 51

22 2 2

2 2 2

9 12

1 2 1 2

10!( 5) 0.5 (0.5) 0.24615!(10 5)!

!Pr ob( 9) (1 )!( )!10!Pr ob( 9) 0.9 (0.1) 0.3874

9!(10 9)!likelihood of observing 5, 9 given 0.5, 0.9

0.2461 0.3874 0.095However li

y n y

Y

nY p py n y

Y

y y p px

= = =−

= = −−

= = =−

= = = == =

1 2 1 2kelihood of observing 5, 9 given 0.1, 0.80.0015 0.2684 0.0004

y y p px

= = = == =

Maximum Likelihood –proportions

Page 50: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Inference

Page 51: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Likelihood ratio/deviance

LLuu = likelihood of unrestricted model with k1 = likelihood of unrestricted model with k1 dfdf

LLrr = likelihood of restricted model with k2 = likelihood of restricted model with k2 dfdf

k2 > k1k2 > k1Restrictions placed on k2Restrictions placed on k2--k1 parametersk1 parameters

typically they are set to zerotypically they are set to zero

2 22 ln( / ) ~u rY L L χ=

Page 52: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Deviance

2 22 1

: 0, 1,..., ( 2 1)

2 ln( / ) ~i

u r k k

Ho b i k k

Y L L χ −

= = −

=

2( ) 2 1E Y k k= −

Page 53: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Iteration 7: log likelihood = -5.26836575008 Convergence achieved after 8 iterations Model 3: Probit estimates using the 24 observations 1-24 Dependent variable: success VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -4.00438 1.45771 -2.747 X 0.612845 0.218037 2.811 0.241462 Model 4: Probit estimates using the 24 observations 1-24 Dependent variable: success VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const 0.000000 0.255832 -0.000 Log-likelihood = -16.6355 Comparison of Model 3 and Model 4: Null hypothesis: the regression parameters are zero for the variables X Test statistic: Chi-square(1) = 22.7343, with p-value = 1.86014e-006 Of the 3 model selection statistics, 0 have improved.

Nb 2ln(Lu/Lr) = 2* (-16.6355- -5.2683) = 22.7343

= Lu

=Lr

2{Lu – Lr]= 2[-5.268 + 16.636] = 22.73

Page 54: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Iteration 3: log likelihood = -2099.98151495 Convergence achieved after 4 iterations Model 2: Logit estimates using the 3110 observations 1-3110 Dependent variable: Dole_Clinton_1 VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -3.41038 0.351470 -9.703 log_urban 25.4951 9.10570 2.800 6.36359 prop_highs 4.96073 0.508346 9.759 1.23820 prop_gradprof 15.1026 2.16268 6.983 3.76961 Model 3: Logit estimates using the 3110 observations 1-3110 Dependent variable: Dole_Clinton_1 VARIABLE COEFFICIENT STDERROR T STAT SLOPE (at mean) const -0.972329 0.184314 -5.275 prop_highs 2.09692 0.360723 5.813 0.523414 Log-likelihood = -2136.12 Comparison of Model 2 and Model 3: Null hypothesis: the regression parameters are zero for the variables log_urban prop_gradprof Test statistic: Chi-square(2) = 72.2793, with p-value = 2.01722e-016

Page 55: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Hants, IoW suburban SE 0.062916 9 11Kent suburban SE 0.035541 4 10Avon suburban not SE 0.133422 4 14Cornwall, Devon rural not SE 0.141939 5 12Dorset, Somerset rural not SE 0.145993 12 16

DATA LAYOUT FOR LOGISTIC REGRESSIONDATA LAYOUT FOR LOGISTIC REGRESSION

REGION URBAN SE/NOT SE OUREGION URBAN SE/NOT SE OUTPUT GROWTH Y nTPUT GROWTH Y n

S Yorks urban not SE -0.150591 0 11W Yorks urban not SE 0.152066 7 15

Page 56: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Logistic Regression Table

Predictor Coef StDev Z PConstant -1.2132 0.3629 -3.34 0.001gvagr 9.716 1.251 7.77 0.000 URBAN/SUBURBAN/RURAL suburban -0.8464 0.2957 -2.86 0.004urban -1.3013 0.4760 -2.73 0.006SOUTH-EAST/NOT SOUTH-EASTSouth-East 2.4411 0.3534 6.91 0.000

Log-Likelihood = -210.068

Page 57: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Testing variables

-247.1 32

-210.068 29

2*Difference = 74.0642*Difference = 74.064

ProbProb = = f(qf(q))

ProbProb = = f(q,urban,SEf(q,urban,SE))

74.064 > 7.81, the critical value equal to the 74.064 > 7.81, the critical value equal to the upper 5% point of the chiupper 5% point of the chi--squared distribution withsquared distribution with3 degree of freedom3 degree of freedomthus introducing URBAN/SUBURBAN/RURALthus introducing URBAN/SUBURBAN/RURALand SE/not SE causes a significantand SE/not SE causes a significantimprovement in fitimprovement in fit

LogLog--likelike degrees of freedomdegrees of freedom

Page 58: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Interpretation• When the transformation gives a linear equation linking

the dependent variable and the independent variables then we can interpret it in the normal way

• The regression coefficient is the change in the dependent variable per unit change in the independent variable, controlling for the effect of the other variables

• For a dummy variable or factor with levels, the regression coefficient is the change in the dependent variable associated with a shift from the baseline level of the factor

Page 59: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Interpretation

)ˆ1/(ˆln( ii PP −

Changes by 9.716 for a unit change in Changes by 9.716 for a unit change in gvagrgvagr

by 2.441 as we move from not SE to SE countieby 2.441 as we move from not SE to SE countiess

by by --1.3013 as we move from RURAL to URBAN1.3013 as we move from RURAL to URBAN

by by --0.8464 as we move from RURAL to SUBURBAN 0.8464 as we move from RURAL to SUBURBAN

Page 60: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

Interpretation

The odds of an event = ratio of Prob(event) to Prob(not event)

The odds ratio is the ratio of two odds.

The logit link function means that parameter estimates are theexponential of the odds ratio (equal to the logit differences).

Page 61: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

InterpretationFor example, a coefficient of zero would indicate thatFor example, a coefficient of zero would indicate thatmoving from a non SE to a SE location produces no changemoving from a non SE to a SE location produces no changein the in the logitlogit

Since exp(0) = 1, this would mean the Since exp(0) = 1, this would mean the (estimated) odds = (estimated) odds = Prob(expand)/Prob(notProb(expand)/Prob(not expand) expand) do not change do not change ieie the odds ratio =1 the odds ratio =1

In reality, since exp(2.441) = 11.49 the odds ratio is 11.49In reality, since exp(2.441) = 11.49 the odds ratio is 11.49The odds of SE firms expanding are 11.49 times theThe odds of SE firms expanding are 11.49 times theodds of non SE firms expandingodds of non SE firms expanding

Page 62: Regression with limited dependent variables - Alumni · Regression with limited dependent variables • In each case, the outcome is binary • We can treat the variable as a success

InterpretationConstant -1.2132 0.3629 -3.34 0.001gvagr 9.716 1.251 7.77 0.000 16584.51 1428.30 1.93E+05RURAL/SUBURBAN/URBAN suburban -0.8464 0.2957 -2.86 0.004 0.43 0.24 0.77urban -1.3013 0.4760 -2.73 0.006 0.27 0.11 0.69SE/not SE SE 2.4411 0.3534 6.91 0.000 11.49 5.75 22.96

Note that the odds ratio has a 95% confidence intervalNote that the odds ratio has a 95% confidence intervalsince 2.4411+1.96*0.3534 =3.1338 since 2.4411+1.96*0.3534 =3.1338 and 2.4411and 2.4411--1.96*0.3534 = 1.74841.96*0.3534 = 1.7484

and exp(3.1338)=22.96 , exp(1.7484) = 5.75and exp(3.1338)=22.96 , exp(1.7484) = 5.75The 95% The 95% c.i.forc.i.for the odds ratio is the odds ratio is 5.75 to 22.965.75 to 22.96

param. est. s.e. t ratio p-value odds ratio lower c.i. upper c.i.