example of models for the study of change david a. kenny december 15, 2013
Post on 20-Jan-2018
213 Views
Preview:
DESCRIPTION
TRANSCRIPT
Example of Models for the Study of Change
David A. Kenny
December 15, 2013
2
Example DataAn Honors Thesis done by Allison Gillum of
Skidmore College supervised by John Berman on the effects of an semester-long class on the Environment on Environmental Responsible Behaviors.
Pretest-Posttest Design41 Treated and 199 Controls2 Treated classes and 8 Control classes. No
clustering effect due to class.Outcome: Environmentally Responsible Behaviors
(ERB), a 12 item scale ranging for 1 to 7. For latent variable analyses, 3 parcels of 4 items were created.
3
Models• Models
– Controlling for Baseline• Simple• Allowing for Unreliability at Time 1
– Change Score Analysis• Raykov• LCS• Kenny-Judd
– Standardized Change Analysis• Types
– Univariate (average of 12 items)– Latent Variable (3 parcels of 4 items)
4
Latent Variable Measurement Models
• Unconstrained– 2(9) = 13.22, p = .153– RMSEA = 0.044; TLI = .992
• Equal Loadings– 2(11) = 18.63, p = .068– RMSEA = 0.054; TLI = .988• The equal loading model has
reasonable fit.
5
Pretest Difference• Mean for Controls: 4.79• Mean for Treateds: 5.21• A mean difference of 0.42• t(238) = 3.191, p = .002• d = 0.64, a moderate effect size• There is a difference at the pretest!• The mean difference on latent variable at time 1 is 0.45.
6
More on the Pretest Difference
• Likely more environmentally conscious students more likely to take an environmental course.
• Would you expect the difference to persist (CSA) or narrow (CfB)?
7
Controlling for Baseline: Univariate
•A beneficial effect of the course on the outcome: 0.3049.• Z = 3.992, p < .001• = 0.776 (expect a narrowing of the gap)•We shall see that this is the largest estimate of
the treatment effect.
8
9
CfB: Measurement Error in the Pretest
Coefficient alpha of .872 for pretestLord-Porter Correction
Convert (.872 - .2112)/( 1 - .2112) = .866Adjusted pretest score (MT is the mean for the Treated and MC is the mean for the controls): (X1 – MT) + MT
(X1 – MC) + MC
b = 0.2569, Z = 3.3308, p < .001
10
11
Williams & Hazer Method
Set X1 = X1T + E1
Fix the variance of E1 to (1 - )sY12
or (1 - .872)(0.614) = 0.079. b = 0.2543, Z = 3.233, p = .001
(with = .897)Same estimates of b and as Lord-
Porter (standard error a bit different)!
12
13
Controlling for Baseline: Latent Variables
–b = 0.3256, Z = 4.124, p < .001– = 0.816 (surprisingly relatively
low)–Cannot directly compare estimates
to the univariate analysis.
14
15
Change Score Analysis: Univariate–All 3 methods (see next 3 slides) show a
beneficial effect:• 0.2105, Z = 2.619, p = .009
–Smallest effect of any analysis,–Note that the Treateds improve (0.0874),
and the Controls decline (-0.1231).
16
Raykov
17
LCS
18
Kenny-Judd
19
Change Score Analysis: Latent Variables
–All 3 methods show a beneficial effect 0.2435, Z = 2.964, p < .003
–Again, you cannot directly compare the univariate and latent variable results.
–Smallest effect of any latent variable analysis.
20Raykov
21LCS
22
KJ
23
Standardized Change Score Analysis: Univariate Analysis
Residual variance decreases slightly over time (but not significantly, p = .31)
•Time 1: 0.59•Time 2: 0.52
Effect: .3078, Z = 2.775 , p = .006Recalibrated to units of Time 2: 0.2266 , Z = 2.826 , p = .005.
24SCSA
25SCSA-Y2
26
Standardized Change Score Analysis: Latent Variables
Residual variance decreases slightly over time (but not significantly, p = .30)
•Time 1: 0.56•Time 2: 0.52
b = 0.3653, Z = 3.116 , p = .002Units of Time 2: b = 0.2627, Z = 3.194 , p = .001
27
SCSA
28SCSA-Y2
29
Summary of Univariate Effects
CfB: 0.3049CfB with Reliability Correction:0.2543CSA: 0.2105SCSA: 0.2266
30
Summary of Latent Variable Effects
CfB: 0.3256CSA: 0.2435SCSA: 0.2627
31
What Estimate Would I Report?CSA Latent Variable: 0.2435(Z = 2.964, p < .003) No reason to think that the factors that created the Time 1 difference to change. Note too the variance does not change.Others might respectfully disagree.
top related