example of models for the study of change david a. kenny december 15, 2013
DESCRIPTION
3 Models –Controlling for Baseline Simple Allowing for Unreliability at Time 1 –Change Score Analysis Raykov LCS Kenny-Judd –Standardized Change Analysis Types –Univariate (average of 12 items) –Latent Variable (3 parcels of 4 items)TRANSCRIPT
![Page 1: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/1.jpg)
Example of Models for the Study of Change
David A. Kenny
December 15, 2013
![Page 2: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/2.jpg)
2
Example DataAn Honors Thesis done by Allison Gillum of
Skidmore College supervised by John Berman on the effects of an semester-long class on the Environment on Environmental Responsible Behaviors.
Pretest-Posttest Design41 Treated and 199 Controls2 Treated classes and 8 Control classes. No
clustering effect due to class.Outcome: Environmentally Responsible Behaviors
(ERB), a 12 item scale ranging for 1 to 7. For latent variable analyses, 3 parcels of 4 items were created.
![Page 3: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/3.jpg)
3
Models• Models
– Controlling for Baseline• Simple• Allowing for Unreliability at Time 1
– Change Score Analysis• Raykov• LCS• Kenny-Judd
– Standardized Change Analysis• Types
– Univariate (average of 12 items)– Latent Variable (3 parcels of 4 items)
![Page 4: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/4.jpg)
4
Latent Variable Measurement Models
• Unconstrained– 2(9) = 13.22, p = .153– RMSEA = 0.044; TLI = .992
• Equal Loadings– 2(11) = 18.63, p = .068– RMSEA = 0.054; TLI = .988• The equal loading model has
reasonable fit.
![Page 5: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/5.jpg)
5
Pretest Difference• Mean for Controls: 4.79• Mean for Treateds: 5.21• A mean difference of 0.42• t(238) = 3.191, p = .002• d = 0.64, a moderate effect size• There is a difference at the pretest!• The mean difference on latent variable at time 1 is 0.45.
![Page 6: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/6.jpg)
6
More on the Pretest Difference
• Likely more environmentally conscious students more likely to take an environmental course.
• Would you expect the difference to persist (CSA) or narrow (CfB)?
![Page 7: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/7.jpg)
7
Controlling for Baseline: Univariate
•A beneficial effect of the course on the outcome: 0.3049.• Z = 3.992, p < .001• = 0.776 (expect a narrowing of the gap)•We shall see that this is the largest estimate of
the treatment effect.
![Page 8: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/8.jpg)
8
![Page 9: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/9.jpg)
9
CfB: Measurement Error in the Pretest
Coefficient alpha of .872 for pretestLord-Porter Correction
Convert (.872 - .2112)/( 1 - .2112) = .866Adjusted pretest score (MT is the mean for the Treated and MC is the mean for the controls): (X1 – MT) + MT
(X1 – MC) + MC
b = 0.2569, Z = 3.3308, p < .001
![Page 10: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/10.jpg)
10
![Page 11: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/11.jpg)
11
Williams & Hazer Method
Set X1 = X1T + E1
Fix the variance of E1 to (1 - )sY12
or (1 - .872)(0.614) = 0.079. b = 0.2543, Z = 3.233, p = .001
(with = .897)Same estimates of b and as Lord-
Porter (standard error a bit different)!
![Page 12: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/12.jpg)
12
![Page 13: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/13.jpg)
13
Controlling for Baseline: Latent Variables
–b = 0.3256, Z = 4.124, p < .001– = 0.816 (surprisingly relatively
low)–Cannot directly compare estimates
to the univariate analysis.
![Page 14: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/14.jpg)
14
![Page 15: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/15.jpg)
15
Change Score Analysis: Univariate–All 3 methods (see next 3 slides) show a
beneficial effect:• 0.2105, Z = 2.619, p = .009
–Smallest effect of any analysis,–Note that the Treateds improve (0.0874),
and the Controls decline (-0.1231).
![Page 16: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/16.jpg)
16
Raykov
![Page 17: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/17.jpg)
17
LCS
![Page 18: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/18.jpg)
18
Kenny-Judd
![Page 19: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/19.jpg)
19
Change Score Analysis: Latent Variables
–All 3 methods show a beneficial effect 0.2435, Z = 2.964, p < .003
–Again, you cannot directly compare the univariate and latent variable results.
–Smallest effect of any latent variable analysis.
![Page 20: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/20.jpg)
20Raykov
![Page 21: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/21.jpg)
21LCS
![Page 22: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/22.jpg)
22
KJ
![Page 23: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/23.jpg)
23
Standardized Change Score Analysis: Univariate Analysis
Residual variance decreases slightly over time (but not significantly, p = .31)
•Time 1: 0.59•Time 2: 0.52
Effect: .3078, Z = 2.775 , p = .006Recalibrated to units of Time 2: 0.2266 , Z = 2.826 , p = .005.
![Page 24: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/24.jpg)
24SCSA
![Page 25: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/25.jpg)
25SCSA-Y2
![Page 26: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/26.jpg)
26
Standardized Change Score Analysis: Latent Variables
Residual variance decreases slightly over time (but not significantly, p = .30)
•Time 1: 0.56•Time 2: 0.52
b = 0.3653, Z = 3.116 , p = .002Units of Time 2: b = 0.2627, Z = 3.194 , p = .001
![Page 27: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/27.jpg)
27
SCSA
![Page 28: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/28.jpg)
28SCSA-Y2
![Page 29: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/29.jpg)
29
Summary of Univariate Effects
CfB: 0.3049CfB with Reliability Correction:0.2543CSA: 0.2105SCSA: 0.2266
![Page 30: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/30.jpg)
30
Summary of Latent Variable Effects
CfB: 0.3256CSA: 0.2435SCSA: 0.2627
![Page 31: Example of Models for the Study of Change David A. Kenny December 15, 2013](https://reader035.vdocuments.us/reader035/viewer/2022070616/5a4d1be27f8b9ab0599e071d/html5/thumbnails/31.jpg)
31
What Estimate Would I Report?CSA Latent Variable: 0.2435(Z = 2.964, p < .003) No reason to think that the factors that created the Time 1 difference to change. Note too the variance does not change.Others might respectfully disagree.