Variability Indicators in Structural Equation Models

Michael BidermanUniversity of Tennessee at Chattanooga

www.utc.edu/Michael-Biderman

For the past few years I’ve investigated the utility of a structural equation model of faking of personality questionnaires, specifically the Big Five.

The model is a CFA containing

1) latent variables representing personality dimensions, and

2) one or more latent variables representing amount of response distortion, i.e., faking.

Background: Modeling Faking of Personality Tests

The Basic Faking Model

F

FE3FE2FE1

FE4FE5

FA3FA2FA1

FA4FA5

FC3FC2FC1

FC4FC5

FS3FS2FS1

FS4FS5

FI3FI2FI1

FI4FI5

E HE3HE2HE1

HE4HE5

A HA3HA2HA1

HA4HA5

C HC3HC2HC1

HC4HC5

S HS3HS2HS1

HS4HS5

I HO3HO2HO1

HO4HO5

E-H

A-H

C-H

S-H

O-H

E-D

A-D

C-D

S-D

O-D

E-I

A-I

C-I

S-I

I-I

E

A

C

S

I

FP

FA

Applied to two-condition data Applied to three-condition data

Beyond the Basic Model:

The basic model represents changes in central tendency associated with faking fairly well.

Is that all there is?

Last year, during manual data entry of Mike Clark’s thesis data (Yes, UTC is a full-service university) . . .I noticed that some participants seemed to be targeting specific responses, e.g., 6 6 6 6 6 6

Since such targeting results in low variability this suggested the possibility that variability of responding might be an indicator of faking.

The following describes an attempt to model variability.

5

Other studies of variability

Traitedness studies (Britt, 1993; Dwight, Wolf & Golden, 2002; Hershberger, Plomin, & Pedersen, 1995).

A person is highly traited on a dimension if the variability of responses to items from the dimension is small.

Extreme response style (Greenleaf, 1992).

Stability of responses to the same scale over time (Eid & Diener, 1999; Kernis, 2005).

No studies of variability of responses in faking situations. None of variability and the Big Five.

1: Biderman & Nguyen, 2004. N=2032: Wrensen & Biderman, 2005. N=166

Two-condition data: Honest and Fake Good50 item IPIP Big Five questionnaire given twice 2-item parcels analyzed

3. Clark & Biderman, 2006. N=166

Three-conditions: Honest, Incentive, Instructed fakingThree 30-item IPIP Questionnaires given.Whole-scale scores analyzed.

Wonderlic Personnel Test (WPT) was given to all participants prior to the first condition.

Three used datasets.

7

Measuring Variability

To represent variability of responses within dimensions,

I computed the standard deviation of responses within each Big Five dimension for each participant for each condition

I added the standard deviations as observed variables to the data to which the faking model had previous been applied

8

Datasets 1 and 2 with Standard Deviations added

F

FE3FE2FE1

FE4FE5

FA3FA2FA1

FA4FA5

FC3FC2FC1

FC4FC5

FS3FS2FS1

FS4FS5

FI3FI2FI1

FI4FI5

E HE3HE2HE1

HE4HE5

A HA3HA2HA1

HA4HA5

C HC3HC2HC1

HC4HC5

S HS3HS2HS1

HS4HS5

I HO3HO2HO1

HO4HO5

SdHE

SdHA

SdHC

SdHS

SdHI

SdFE

SdFA

SdFC

SdFS

SdFI

9

Dataset 3 with standard deviations added

E-H

A-H

C-H

S-H

O-H

E-D

A-D

C-D

S-D

O-D

E-I

A-I

C-I

S-I

I-I

E

A

C

S

I

FP

FA

SdE-H

SdA-H

SdC-H

SdS-H

SdO-H

SdE-D

SdA-D

SdC-D

SdS-D

SdO-D

SdE-I

SdA-I

SdC-I

SdS-I

SdO-I

10

Modeling Standard Deviations - 1

Faking leads to elevated central tendency, often resulting in ceiling effects.

Ceiling effects lead to lower variability.

So the standard deviations were connected to central tendency via regression links.

Specifically, standard deviations were regressed onto parcel or scale scores.

11

Modeling Ceiling Effects in Datasets 1 and 2:Standard deviations were regressed onto parcel scores

F

FE3FE2FE1

FE4FE5

FA3FA2FA1

FA4FA5

FC3FC2FC1

FC4FC5

FS3FS2FS1

FS4FS5

FI3FI2FI1

FI4FI5

E HE3HE2HE1

HE4HE5

A HA3HA2HA1

HA4HA5

C HC3HC2HC1

HC4HC5

S HS3HS2HS1

HS4HS5

I HO3HO2HO1

HO4HO5

SdHE

SdHA

SdHC

SdHS

SdHI

SdFE

SdFA

SdFC

SdFS

SdFI

12

Modeling Ceiling Effects in Dataset 3: Standard Deviations regressed onto scale scores

E-H

A-H

C-H

S-H

O-H

E-D

A-D

C-D

S-D

O-D

E-I

A-I

C-I

S-I

I-I

E

A

C

S

I

FP

FA

SdE-H

SdA-H

SdC-H

SdS-H

SdO-H

SdE-D

SdA-D

SdC-D

SdS-D

SdO-D

SdE-I

SdA-I

SdC-I

SdS-I

SdO-I

13

Modeling Standard Deviations - 2

The assumption/hope? was that there are individual differences in variability of responding within dimensions

So a latent variable representing such individual differences was added to the model.

14

Modeling variability in Dataset 1 & 2: Adding a “Variability” latent variable

F

FE3FE2FE1

FE4FE5

FA3FA2FA1

FA4FA5

FC3FC2FC1

FC4FC5

FS3FS2FS1

FS4FS5

FI3FI2FI1

FI4FI5

.

.

.

.

.

E HE3HE2HE1

HE4HE5

A HA3HA2HA1

HA4HA5

C HC3HC2HC1

HC4HC5

S HS3HS2HS1

HS4HS5

I HO3HO2HO1

HO4HO5

.

.

.

.

.

SdHE

SdHA

SdHC

SdHS

SdHI

SdFE

SdFA

SdFC

SdFS

SdFI

V

15

Modeling variability in Dataset 3: Adding a “Variability” latent variable

E-H

A-H

C-H

S-H

O-H

E-D

A-D

C-D

S-D

O-D

E-I

A-I

C-I

S-I

I-I

E

A

C

S

I

FP

FA

SdE-H

SdA-H

SdC-H

SdS-H

SdO-H

SdE-D

SdA-D

SdC-D

SdS-D

SdO-D

SdE-I

SdA-I

SdC-I

SdS-I

SdI-I

V

16

Results

Did the regression links significantly improve goodness of fit?

Are there individual differences in variability of responding that are captured by the V latent variable?

17

Results of application of Variability model to Dataset 1Model Χ2(1539)=2501.249; p<.001; CFI=.871; RMSEA=.055

Both the regression links and the V latent variable improved model fit.

F

.50

.60

.62

.44

.50

FE3FE2FE1

FE4FE5

FA3FA2FA1

FA4FA5

FC3FC2FC1

FC4FC5

FS3FS2FS1

FS4FS5

FI3FI2FI1

FI4FI5

.37

.30

.27

.31

.22

E HE3HE2HE1

HE4HE5

A HA3HA2HA1

HA4HA5

C HC3HC2HC1

HC4HC5

S HS3HS2HS1

HS4HS5

I HI3HI2HI1

HI4HI5

.26

.37

.31

.27

.50

.10

.16

.28

.34

.38

.81

.69

.72

.79

.60

SHE-.06

.08

.35

.08

.30.17

SHA-.14

SHC-.14

SHS-.07

SHI-.11

SFE-.07

SFA-.16

SFC-.16

SFS-.09

SFI-.09

V

.31

.52

.47

.36

.38

.27

.42

.34

.34

.45

ΔΧ2(31)=597.537p<.001

Chi-square difference test of regression links: Χ2(50)=625.685p<.001

Chi-square difference test of V latent variable:Χ2(16)=218.041p<.001

18

Results of application of Variability model to Dataset 2Model Χ2(1539)=2666.874; p<.001; CFI=.816; RMSEA=.066

Chi-square difference test of V latent variable:ΔΧ2(16)=274.468p<.001

ΔΧ2(31)=608.423p<.001

F

.56

.63

.67

.51

.50

FE3FE2FE1

FE4FE5

FA3FA2FA1

FA4FA5

FC3FC2FC1

FC4FC5

FS3FS2FS1

FS4FS5

FO3FO2FO1

FO4FO5

.26

.20

.19

.26

.25

E HE3HE2HE1

HE4HE5

A HA3HA2HA1

HA4HA5

C HC3HC2HC1

HC4HC5

S HS3HS2HS1

HS4HS5

I HO3HO2HO1

HO4HO5

.14

.23

.23

.26

.20

-.07

.01

.02

.28

.31

.75

.67

.70

.75

.63

SHE-.08

.02

.00

.05

-.01.08

SHA-.16

SHC-.14

SHS-.06

SHO-.14

SFE-.13

SFA-.20

SFC-.19

SFS-.09

SFO-.18

V

.52

.39

.35

.38

.42

.56

.36

.40

.25

.46

Chi-square difference test of regression links:ΔΧ2(50)=747.249p<.001

Again, both the regression links and the V latent variable improve model fit.

19

Results of application of Variability model to Dataset 3Model Χ2(352)=532.552; p<.001; CFI=.883; RMSEA=.056

E-H

A-H

C-H

S-H

O-H

E-D

A-D

C-D

S-D

O-D

E-I

A-I

C-I

S-I

I-I

E

A

C

S

I

FP

FA

SdE-H

SdA-H

SdC-H

SdS-H

SdO-H

SdE-D

SdA-D

SdC-D

SdS-D

SdO-D

SdE-I

SdA-I

SdC-I

SdS-I

SdI-I

V

43

.26

.11

.26

.19

.35

.16

.26

.31

.57

.73

.62

.29

.38

-.09

-.24

-.43

.47

.52

.41

.67

Chi-square difference test of V latent variable:ΔΧ2(22)=405.333p<.001

ΔΧ2(12)=37.910p<.05

ΔΧ2(17)=260.217p<.001(FP correlations with Big 5 LVs set at 0 for this test.)

Chi-square difference test of regression links:ΔΧ2(15)=268.367p<.001

20

Tentative Conclusions regarding Variability Model

1) Ceiling effects seem to be successfully modeled by the regressions of standard deviations onto parcels or scale scores.

2) Individual differences in variability of responding to items within dimensions seem to be captured by the V latent variable.

Some persons consistently exhibited little variability in responding to items within questionnaire scales.

Others exhibit greater variability in responding.

21

What about V and faking?1) Loadings on V might be larger in faking conditions – magnifying individual differences in variability because some people are targeting while others are not.

Mean Standardized Loadings of Standard Deviation indicators on V

Dataset 1 Honest Incentive to fake Instructed to fakeMean loading .406 .366

Dataset 2 Honest Incentive to fake Instructed to fakeMean loading .411 .496

Dataset 3 Honest Incentive to fake Instructed to fakeMean loading .468 .521 .413

Tentative Conclusion: Loadings on V are approximately equal in honest and faking conditions.

22

2) V might be related to the faking latent variables.

Dataset 1: Correlation of V with F: .04 NS

Dataset 2: Correlation of V with F: .02 NS

Dataset 3: Correlation of V with FP: .16 NS

Correlation of V with FA: .10 NS

It appears from these preliminary analyses that variability of responding is not related to faking

However, other scenarios and models should be explored

What about V and faking?

23

What is V?1) Perhaps V is related to personality characteristics

It appears that V has discriminant validity with respect to the Big Five.

24

What is V?2) Perhaps V is related to cognitive ability

These results suggest that persons with higher CA exhibit less variability of responding.

25

Uses of VHow about using it to extract cognitive ability from the Big Five?

FA

V

I

WPT

e Dataset 1: Multiple R = .57

Dataset 2: Multiple R = .35

Dataset 3: Multiple R = .50

The structural model suggests that there is information on cognitive ability embedded in “noncognitive” personality tests.

26

Conclusions

1) V appears to be an individual difference variable that cuts across personality dimensions.

2) V appears to be unrelated to faking

3) V appears to be independent of the Big Five dimensions.

4) V appears to be related to cognitive ability – persons higher in cognitive ability have lower variability of responding

27

Implications

Don’t throw away old datasets.

You never know what constructs may be hidden in them.