the university of tennessee at chattanooga | university ... · web viewextraversion, agreeableness,...

22
Lecture 13 – Applications of EFA, CFA, Bifactor Models and Factor Scores The Big Five Five general traits believed to be the foundation characteristics of normal personality Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades of research, the field has now achieved an initial consensus on a general taxonomy of personality traits, the “Big Five” personality dimensions.” (John, O. P. (2008) in John, Robins, & Pervin, Handbook of Personality: Theory and Research, 3 rd Ed.) Path Diagrams of the Big Five

Upload: others

Post on 10-Oct-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Lecture 13 – Applications of EFA, CFA, Bifactor Models and Factor Scores

The Big Five

Five general traits believed to be the foundation characteristics of normal personality

Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience

“After decades of research, the field has now achieved an initial consensus on a general taxonomy of personality traits, the “Big Five” personality dimensions.” (John, O. P. (2008) in John, Robins, & Pervin, Handbook of Personality: Theory and Research, 3rd Ed.)

Path Diagrams of the Big Five

The HEXACO

The Big Five (although with indicators reflecting different takes on Agreeableness and Stability) plusHonesty/Humility

Each specific domain is indicated by 2 – 20+ items, usually about 10.

The content of each item is supposed to be specific to the domain.

For example, the Extraversion items are supposed to reference only extraversion, nothing else.

Same for each of the other domains

Note that the path diagrams are separate, reflecting the belief / hope that the factors are orthogonal with no cross loadings between factors and items from different domains.

Page 2: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Key Assumptions of the Big Five and HEXACO

Everyone possesses each trait, although different people possess different profiles

Dimensions are orthogonal

Being high on one dimension doesn’t mean you’ll be high or low on any other

It’s possible to be high on all; low on all; high on only one; high on two; etc.

Two Major Issues with respect to the Big Five and HEXACO

1. Summated Scales are invariably positively correlated

Block, J. (1995). A contrarian view of the five-factor approach to personality description. Psychological Bulletin, 117, 187-215. doi:10.1037/0033-2909.117.2.187

“. . . the empirical research findings indicate that the five factors are frequently and importantly correlated with each other.”

Examples of this issue from UTC data . . .

Page 3: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Issue 2 – Big Five / HEXACO Summated Scales correlations with measures of affect states

UTC Scale Correlations with Rosenberg Self-esteem Scale

Boldfaced correlations are significant at alpha = .05.

Questionnaire Extrav Agree Conscien Stabil Openness Hon/Hum

NEO-FFI; N=1195 .51 .28 .53 .66 .03

IPIP 50; N=531 .26 .12 .28 .30 .23

BFI-2; N=638 .44 .18 .41 .60 .18

HEX 100; N=1195 .64 .20 .40 .12 -.01 .15

HEX 60; N=1373 .61 .11 .29 .18 -.04 .14

UTC Scale Correlations with Costello & Comrey Depression Scale score as Depression.

NEO-FFI; N=1195 -.56 -.42 -.51 -.54 -.06

IPIP 50; N=531 -.22 -.17 -.19 -.29 -.11

HEX 100; N=1195 -.61 -.25 -.42 .01 .01 -.25

UTC Scale Correlations with PANAS PA Scale

NEO-FFI; N=1195 .60 .29 .59 .46 .14

HEX 100; N=1195 .61 .18 .48 .06 .10 .15

UTC Scale Correlations with PANAS NA Scale

NEO-FFI; N=1195 -.42 -.31 -.43 -.72 .00

HEX 100; N=1195 -.52 -.35 -.32 -.29 .01 -.19

Conclusions

Issue 1. The summated scale scores of the Big 5 and HEXACO are generally positively correlated with each other.

Issue 2. Most of the Big 5 and HEXACO summated scale scores are correlated positively with measures of positive affect and negatively with measures of negative affect.

Page 4: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Hypotheses to explain the two issues

1. The Correlations are Real Hypothesis. The Big Five and HEXACO ARE correlated constructs in the population and all of the dimensions ARE correlated with affect in the population.

2. The Random Correlations Hypothesis. The correlations are the results of random fluctuations in estimated correlations.

3. The Fake News Hypothesis. The correlations are the result of manipulative psychologists lying about the correlations to further a nefarious scheme to make us think that all human characteristics are correlated. After all, we ARE all human, aren’t we?

4. The Common Evaluative Content Hypothesis. All questionnaire items have some evaluative aspect of content in common. This causes responses to all items to be slightly correlated, depending on the amount of that common content.

The common content is in addition to the domain or ostensible content of the items.

The sensitivity to that extra content affects responses to all items.

Persons with positive sensitivity tend to agree with all items to the extent to which the items possess it. Happy people agree with items with positive evaluative content, i.e., positive valence.

Persons with a negative sensitivity tend to disagree with all items to the extent to which the items possess it. Unhappy people disagree with items with negative evaluative content, i.e., negative valence.

Persons with little sensitivity do not tend to agree or disagree with items base on the extent to which the items possess the common content.

Accounting for Issue 1

For example, if Extraversion and Agreeableness items both contain some common content, persons sensitive to the presence of that common content with agree or disagree with both E and A items based on the type and strength of their sensitivity and the amount of the common content.

Accounting for Issue 2

If the common content is evaluative – related to affect – then the presence of evaluative content in items could explain why Big Five and HEXACO measures are related to measures of affect, all of which have considerable evaluative content.

Note that if the common content is NOT evaluative, then some other hypothesis would have to be put forward to account for Issue 2.

So, the Common Evaluative Content Hypothesis could account for both Issue 1 and Issue 2 facing Big Five and HEXACO theorists.

Page 5: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Testing the Common Content Hypothesis

1. Determine if there is a factor indicated to common content. Issue 1.

2. Determine if the content indicating that factor is evaluative content. Issue 2.

Bifactor Models

The general factor in a bifactor model is sensitive to the content of ALL items.

If bifactor models fit the data better than models without a general factor, that is evidence for the existence of a factor indicated by common content. It’s evidence explaining Issue 1.

Test of Issue 1 – Is there a factor indicated by common content?

Chi-square difference comparing CFA Correlated Factors model with Bifactor model

Biderman, Nguyen, Cunningham, & Ghorbani, 2011, p. 422, Table 2.

Questionnaire: IPIP IPIP IPIP IPIP IPIP NEO-FFIChi-square Diff: 272.7 299.2 459.3 672.9 327.11 326.5df: 50 50 50 50 50 60p: <.001 <.001 <.001 <.001 <.001 <.001

Chi-square difference comparing EFA models without and with an added factor

Biderman, McAbee, Chen, & Hendy, 2018, p. 379, Table 2

Questionnaire: NEO-FFI-3 HEXACO 100Chi-square Diff: 775.60 786.53df: 55 90p: <.001 <.001

Other investigators have compared the goodess-of-fit of models without a general factor with models that include a general factor, i.e., bifactor models. All have found significant differences in goodness-of-fit.

So, the bifactor model passes the first test – there is a factor indicated by common content.

Page 6: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Test of Issue 2 – Is the common content that indicates the general factor evaluative?

Correlations of the general factor with PANAS PA and NABiderman, Nguyen, Cunningham, & Ghorbani, 2011, p. 424, Table 7.

Correlation of General factor with PA = .46, p < .001Correlation of General factor with NA = -.36, p < .001

Correlations of loadings of items on the general factor with item evaluative contentBiderman, McAbee, Chen, & Hendy, 2018

A loading is the extent to which a factor influences the response to an item. These figures suggest that the general factor has the greatest effect on items with extreme valence – high positive valence or high negative valence.

For me, the above figures represent key evidence. They depict the relation between two independently obtained aspect of items – valence ratings obtained in 2012 and loadings obtained from data gathered from different people in 2014-2016.

Correlations of general factor with measures of self-esteem, PA, depression, and NABiderman, McAbee, Chen, & Hendy, 2018

Affect MeasureRosenberg RSE PANAS PA C&C Dep PANAS NA

NEO-FFI-3 .59 .53 -.50 -.60HEXACO 100 .28 .36 -.44 -.12

The above data suggest that 1) the characteristic of items that connects them to the general factor is the items evaluative content, i.e., item valence, and2) the general factor is related to measures of affect.

Thus, the above data are evidence for the believe that item evaluative content is the correct aspect of the Common Evaluative Content Hypothesis.

I have a flash-bulb memory of first seeing these correlations in 2009. I couldn’t believe my eyes. A method factor (as we called it then) was correlated with measures of affect!!!

Page 7: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

More on what is in the Big Five and the HEXACO – from a paper under review

In our 2018 paper we found that the general factor in bifactor models seems to be a measure of respondent affect.

We wondered whether that newly unearthed aspect would serve as a predictor of any useful criterion.

The only useful criterion we had was GPA, so we used it.

We applied EFA bifactor models to data of three different questionnaires responded to by three different samples of persons. We compared validities of scale scores, factor scores, and factors in predicting GPA.

The simple validity coefficients of Big Five and HEXACO summated scores

Sample Extrav Agreeable Conscien Stabil Open Hon/Hum

NEO-FFI-3;N=1377 .03 .14 .23 .02 .02

HEXACO 60; n=763 .08 .01 .28 -.04 .03 .12

BFI-2; n=916 -.01 .10 .15 .02 .05

Major findings . . .

1. Big Five and HEXACO domain scale scores are, with one exeption, not strongly related to GPA.

2. Conscientiousness IS a valid predictor of GPA across these three questionnaires.

The simple validity of Big Five and HEXACO factor scores from an EFA bifactor model

We applied an EFA bifactor model, estimating a general factor along with domain factors and an acquiescence factor. We computed factor scores for each factor and input those factor scores into SPSS.

Here are the simple validity coefficients of the domain and general factor scores. Red’d are p < .05.

Sample Extrav Agreeable Conscien Stabil Open Hon/Hum General

NEO-FFI-3;N=1377 -.09 .00 .18 -.09 .00 .20HEXACO 60; n=763 -.03 -.05 .24 -.03 .01 .07 .15BFI-2; n=916 -.08 -.01 .17 -.07 -.01 .20

Major findings . . .

1. The General factor is a valid predictor of GPA for each questionnaire with largest validity in two of the three samples.

2. Conscientiousness remains the only domain score that is consistently valid.

Page 8: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Major findings continued . . .

3. Validity of the factor score measures of the domains are consistently more negative than validity of the scale score measures.

Repeat of the table with both Scale score and Factor score validities. Red’d are p < .05.

Sample Extrav Agreeable Conscien Stabil Open Hon/Hum General

NEO-FFI-3 Scale .03 .14 .23 .02 .02NEO-FFI-3 Factor -.09 .00 .18 -.09 .00 .20

HEXACO 60 Scale .08 .01 .28 -.04 .03 .12HEXACO 60; Factor -.03 -.05 .24 -.03 .01 .07 .15

BFI-2 Scale -.01 .10 .15 .02 .05BFI-2 Factor -.08 -.01 .17 -.07 -.01 .20

Every factor score validity coefficient with the exception of the shaded pair is significantly more negative than its scale score equivalent.

Our conclusion . . .

Removing the evaluation from the items to put into the general factor removed it from the scale scores and their factors.

Since evaluation is positively correlated with GPA, removing that positively correlated predictor made the validity of what was left less positive.

G

Page 9: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Detail Work - CFA analysis replicating the above

We applied CFA bifactor models to the same data. We computed factor scores from the CFA models using the same technique that we’ve used with lavaan.

Here is the table of simple validity coefficients of the scale scores and factor scores for each questionnaire . . .

In this figure, boldfaced correlations are significantly different from each other.

The fourth set of validity coefficients is from a separate analysis that is not discussed here.

.20

.15

.20

From EFA Factor scores

Page 10: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Extending the results - Replicating the results using multiple regression analyses of scale scores, factor scores, and factors.

The first line for each sample gives partial regression coefficients (betas) for a multiple regression of GPA onto the domain summated scores.

The second line gives partial regression coefficients (betas) for a multiple regression of GPA onto EFA (ESEM) factor scores for a bifactor model estimating a general factor (labelled Evaluation in the table) and an acquiescence factor (not elaborated upon here.)

Since we were using Mplus, we were also able to analyze the relation of GPA to all factors using SEM techniques from an EFA perspective – called ESEM in Mplus. Those results are presented in the third line for each sample in the following table.

Major findings from this table. . .

1. The general (labelled Evaluation) factor was the best unique predictor in two of the three samples – better than conscientiousness, just as above.

2. The same pattern of results – scale score partial regression coefficients were smaller than factor score regression coefficients – was found here as was found above for simple validity coefficients.

3. ESEM (Exploratory SEM) partial regression coefficients were all stronger than the corresponding factor score regression coefficients, as were the ESEM R-squareds.

4. If you’re interested, Acquiescence was not a strong predictor of GPA.

Page 11: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

My conclusions from this lecture . . .

1. Evaluative content is present in different degrees in all personality questionnaire items.

2. If a respondent is instructed to report the extent to which the behavior represented by items represents himself or herself, the response will be influenced by item evaluative content.

1+2. Evaluative content is there and it affects every response.

3. The overall effect of evaluative content can be assessed using bifactor modeling techniques.

4. The affective state of a respondent can be estimated by computing factor scores from application of a bifactor model.

3+4. We can measure and use the effect of evaluative content in models of questionnaire responses.

Page 12: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Graphical depiction of the portioning of variance in bifactor models – copied from Lecture 12.Biderman – Master Tutorial on Bifactor Models, SIOP 2013.

Variance portions associated with a bifactor model

The colored part of each rectangle is the portion of of variance due to the influence of the item’s Big Five trait

The white part of each square is the portion of variance due to the influence of error of measurement (latent variables not shown in figure).

The black part of each square is the portion of variance due to the influence of the general factor.

GFP

O

S

C

A

E

Page 13: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Scale scores if this is the appropriate representation

Freedom from Contamination if this is appropriate representation

Extraversion + General +Error

Extraversion Scale Score

Extraversion Items

Purer Openness

GFP

O

S

C

A

EPurer Extraversiontion

Purer Agreeablenesstion

Purer Conscientiousness

Purer Stability

Pure Whatever the general factor is

Page 14: The University of Tennessee at Chattanooga | University ... · Web viewExtraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience “After decades

Path Diagram for Factor scores

This figure emphasizes that factor scores are NOT the factors, they are estimates of the factors, subject to their own errors of measurement.

The advantage of factor scores is that they are unaffected by

1) item error of measurement, and 2) contamination by other factors.

So, while they are only estimates, they are “purer” estimates than scale scores, which are affected.

GFP

O

S

C

A

E

FS MeasError 6G Factor Scores

FS MeasError 5O Factor Scores

FS MeasError 4S Factor Scores

FS MeasError 3C Factor Scores

FS MeasError 2A Factor Scores

FS MeasError 1E Factor Scores