faking in personnel selection: does it matter and can we do anything about it? eric d. heggestad...

Faking in personnel Faking in personnel selection:selection:

Does it matter and can we do Does it matter and can we do anything about it?anything about it?

Eric D. HeggestadEric D. HeggestadUniversity of North Carolina - CharlotteUniversity of North Carolina - Charlotte

Education Testing Service Mini-Education Testing Service Mini-ConferenceConference

Oct 13Oct 13th th & 14& 14thth 20062006

Four Questions About Four Questions About Faking in Personnel Faking in Personnel Selection ContextsSelection Contexts

1.1. Can people fake?Can people fake?

2.2. Do applicants fake? Do applicants fake?

3.3. Does faking matter?Does faking matter?— I will talk about one projectI will talk about one project

4.4. What do we do about it?What do we do about it?— I will talk about one projectI will talk about one project

Does faking matter?Does faking matter?

Effects on Validity and Effects on Validity and SelectionSelection

Mueller-Hanson, Heggestad, & Thornton Mueller-Hanson, Heggestad, & Thornton (2003)(2003)

Ss completed personality and criterion Ss completed personality and criterion measures in lab settingmeasures in lab setting

Personality measurePersonality measure— Achievement Motivation InventoryAchievement Motivation Inventory

Criterion measureCriterion measure— A speeded ability test with no time limitA speeded ability test with no time limit— Could leave when they wanted, opportunity for Could leave when they wanted, opportunity for

normative feedbacknormative feedback

GroupsGroups— Honest Honest ((nn = 240) = 240) vs. faking vs. faking ((nn = 204) = 204)

Means & Standard Means & Standard DeviationsDeviations

PredictorPredictor

CriterionCriterion

Faking Faking GroupGroup

Honest Honest GroupGroup

Effect Effect SizeSize

214.7 225.6 0.41

40.5 40.1 -0.05

Criterion-Related Criterion-Related ValidityValidity

Faking Faking GroupGroup

Honest Honest GroupGroup

.17* .05

* p < .05

Upper thirdUpper third .20*

Lower thirdLower third .26*

.07

.45*

Full Groups

But Validity is Only But Validity is Only Skin DeepSkin Deep

Important to look at selectionImportant to look at selection Groups were combined and various Groups were combined and various

selection ratios examinedselection ratios examined

Variables examinedVariables examined Percent of selectees from each groupPercent of selectees from each group Performance of those selectedPerformance of those selected

Effects on SelectionEffects on SelectionPercent hired at various selection ratiosPercent hired at various selection ratios

0

10

20

30

40

50

60

70

90 80 70 60 50 25 20 15 10

HonestFaking

Selection Ratio (%)Selection Ratio (%)

Per

cent

of

Sel

ecte

esP

erce

nt o

f S

elec

tees

Note: Honest made up 54% Note: Honest made up 54% of sampleof sample

Effects on SelectionEffects on SelectionGroup performance at various selection Group performance at various selection

ratiosratios

36

37

38

39

40

41

42

43

44

45

46

90 80 70 60 50 25 20 15 10

HonestFaking

Selection Ratio (%)Selection Ratio (%)

Per

form

ance

Per

form

ance

.56.56.50.50.31.31.23.23.18.18.15.15.08.08.09.09.07.07

ConclusionsConclusions

Faking appears to have…Faking appears to have… An impact on the criterion-related An impact on the criterion-related

validity of our predictorvalidity of our predictor— Most noticeably at the high end of the distributionMost noticeably at the high end of the distribution

An impact on the quality of decisionsAn impact on the quality of decisions— Low performing fakers more likely to be selected in Low performing fakers more likely to be selected in

top-down contextstop-down contexts

What do we do about What do we do about faking?faking?

What Do We Do About What Do We Do About Faking?Faking?

Approach 1: Detection and CorrectionApproach 1: Detection and Correction

Tries to correct faking that has already Tries to correct faking that has already occurredoccurred

Score correctionsScore corrections— Not successful (Ellingson, Sackett & Hough, 1999; Not successful (Ellingson, Sackett & Hough, 1999;

Schmitt & Oswald, 2006)Schmitt & Oswald, 2006)

IRT workIRT work RetestingRetesting

What Do We Do About What Do We Do About Faking?Faking?

Approach 2: PreventionApproach 2: Prevention

Many prevention strategiesMany prevention strategies WarningsWarnings Subtle itemsSubtle items Multidimensional forced-choice (MFC) Multidimensional forced-choice (MFC)

response formatsresponse formats

What is an MFC What is an MFC Format?Format?

Dichotomous quartet formatDichotomous quartet format Item contains four statementsItem contains four statements Each statement represents a different traitEach statement represents a different trait 2 statements positively worded, 2 statements positively worded,

2 statements negatively worded2 statements negatively worded Indicate “Most Like Me” and “Least Indicate “Most Like Me” and “Least

Like Me” Like Me”

Example MFC ItemExample MFC Item

Avoid difficult reading material (-)

Only feel comfortable with friends (-)

Believe that others have good intentions (+)

Make lists of things to do (+)

XXXX

XXXX

Most Most Like MeLike Me

Least Least Like MeLike Me

MFC FormatsMFC Formats

Appears to be faking resistant Appears to be faking resistant (Christiansen et al., 1998; Jackson et al., 2000)(Christiansen et al., 1998; Jackson et al., 2000)

Example from Jackson et al. (2000)Example from Jackson et al. (2000) Likert-type format effect size = .95Likert-type format effect size = .95 MFC format effect size = .32MFC format effect size = .32

However….However….

Normative vs. IpsativeNormative vs. Ipsative MFC measures typically provide MFC measures typically provide partiallypartially ipsativeipsative measurement measurement

Selection settings require normative Selection settings require normative assessmentassessment

Also, evaluations have focused on group Also, evaluations have focused on group level analyseslevel analyses

Forced-Choice as Forced-Choice as Prevention? Prevention? Heggestad, Heggestad,

Morrison, Reeve & McCloy (2006)Morrison, Reeve & McCloy (2006)

Two studiesTwo studies Study 1 – Do MFC measures provide Study 1 – Do MFC measures provide

normative trait information?normative trait information?

Study 2 – Are MFC measures resistant to Study 2 – Are MFC measures resistant to faking at individual level?faking at individual level?

Study 1 Study 1 Do MFC measures provide Do MFC measures provide

normative information?normative information?

Participants Participants (n= 307)(n= 307) completed three completed three measures under honest instructionsmeasures under honest instructions

NEO-FFINEO-FFI IPIP Likert measure IPIP Likert measure IPIP MFC measureIPIP MFC measure

— Conducted three data collections to create this Conducted three data collections to create this measuremeasure



Logic: If MFC provides normative Logic: If MFC provides normative information, then correspondence information, then correspondence between …between …

IPIP-Likert and IPIP-MFC scales should IPIP-Likert and IPIP-MFC scales should be quite goodbe quite good

Each IPIP measure and the NEO-FFI Each IPIP measure and the NEO-FFI should be similar should be similar



.81.81

.87.87

.75.75

.75.75

.83.83

.68.68

.67.67

.76.76

.70.70

.81.81

.59.59

.58.58

.65.65

.64.64

.71.71

IPIP Likert IPIP Likert IPIP MFCIPIP MFC

NEO NEO IPIP LikertIPIP Likert

NEO NEO IPIP MFCIPIP MFC

StabilityStability

ExtroversionExtroversion

OpennessOpenness

AgreeablenessAgreeableness

Conscientious.Conscientious.

CorrelationsCorrelations



We also defined correspondence as mean We also defined correspondence as mean percentile differences across the percentile differences across the measuresmeasures

n

tileFORM

tileFORM |%%| 21



14.0014.00

11.3811.38

15.2215.22

16.3916.39

12.6112.61

18.2918.29 21.1321.13

18.6118.61 20.4920.49

15.2815.28 18.5818.58

17.6317.63 19.3119.31

14.0714.07 16.9616.96




StabilityStability


OpennessOpenness



Percentile RankPercentile Rank



ConclusionsConclusions MFC seems to do a reasonable job of MFC seems to do a reasonable job of

capturing normative trait informationcapturing normative trait information— People can be compared directly!People can be compared directly!

Study 2 Study 2 Are MFC measures Are MFC measures

resistant to faking at individual level?resistant to faking at individual level?

Participants Participants (n= 282)(n= 282) completed three completed three measuresmeasures

NEO-FFI NEO-FFI Honest instructions Honest instructions IPIP Likert IPIP Likert Faking instructions Faking instructions IPIP MFC IPIP MFC Faking instructions Faking instructions

Replication of Previous Replication of Previous FindingsFindings

IPIP LikertIPIP Likert IPIP MFCIPIP MFC

StabilityStability


OpennessOpenness



0.750.75

0.650.65

0.360.36

0.650.65

1.231.23

0.610.61

0.330.33

0.130.13

0.070.07

1.201.20

Effect SizesEffect Sizes



Logic: If MFC is resistant to faking at Logic: If MFC is resistant to faking at the individual level, then…the individual level, then…

NEO-FFI NEO-FFI (honest)(honest) IPIP-MFC IPIP-MFC (like honest)(like honest)

andand NEO-FFI NEO-FFI (honest)(honest) IPIP-Likert IPIP-Likert (fakeable)(fakeable)

IPIP-MFC IPIP-MFC IPIP-Likert IPIP-Likert



.62.62

.61.61

.59.59

.48.48

.68.68

.37.37

.37.37

.53.53

.50.50

.40.40

.26.26

.36.36

.55.55

.52.52

.39.39




StabilityStability


OpennessOpenness



CorrelationsCorrelations



20.2320.23

21.0921.09

20.4420.44

24.3324.33

18.0518.05

25.2925.29

24.2324.23

21.8521.85

21.5421.54

23.4723.47

28.8728.87

26.1226.12

20.6920.69

22.8222.82

23.7523.75




StabilityStability


OpennessOpenness



Percentile RankPercentile Rank



Conclusion Conclusion MFC not a solution to fakingMFC not a solution to faking

— Can fake specific scalesCan fake specific scales— Not faking resistant at individual levelNot faking resistant at individual level

Summary and Summary and ConclusionConclusion

Faking does impact scoresFaking does impact scores Changes the nature of the scoreChanges the nature of the score Not likely to have a big effect on CRVNot likely to have a big effect on CRV Could have notable implications for Could have notable implications for

selectionselection

Dichotomous quartet response format Dichotomous quartet response format does not offer a viable remedydoes not offer a viable remedy

faking in personnel selection: does it matter and can we do anything about it? eric d. heggestad...

Documents

faking n

project slide

criterion measures

personnel selection

selection muellerhanson

speeded ability test

time limit