tools for successful selection employment testing: general
TRANSCRIPT
Tools for Successful Tools for Successful SelectionSelection
Employment Testing: GeneralEmployment Testing: General
Types of Employment TestsTypes of Employment Tests
Ability TestsAbility Tests• Mental (Cognitive) AbilityMental (Cognitive) Ability• Psychomotor AbilityPsychomotor Ability• Physical AbilityPhysical Ability• Sensory AbilitySensory Ability
Achievement and Skill TestsAchievement and Skill Tests• Job KnowledgeJob Knowledge• Performance/Work SamplePerformance/Work Sample
Types of Employment TestsTypes of Employment Tests
Personality/Interest InventoriesPersonality/Interest Inventories Other Types of TestsOther Types of Tests
• Integrity (honesty)Integrity (honesty)• DrugDrug
Selecting Employment TestsSelecting Employment Tests Which KSAs are relevant to testing?Which KSAs are relevant to testing? Search sources of testsSearch sources of tests
• GeneralGeneral Tests: A Comprehensive ReferenceTests: A Comprehensive Reference...... Web-based information sources
• ReviewsReviews Mental Measurement Yearbook
• Useful Useful article on using the MMYon using the MMY British Psychological Society Test CritiquesTest Critiques
Evaluating Test ManualsEvaluating Test Manuals
Evidence of reliabilityEvidence of reliability Relevant evidence of validityRelevant evidence of validity
• Use of appropriate criteriaUse of appropriate criteria• Norms, based on appropriate samplesNorms, based on appropriate samples
Evidence of test fairnessEvidence of test fairness PracticalityPracticality
Tools for Successful Tools for Successful SelectionSelection
Cognitive Ability (Intelligence) Cognitive Ability (Intelligence)
TestsTests
What is “Intelligence”?What is “Intelligence”?
And is it relevant And is it relevant to job to job performance??performance??
““Successful” IntelligenceSuccessful” Intelligence
Analytical intelligenceAnalytical intelligence• ““Academic” intelligenceAcademic” intelligence
Practical IntelligencePractical Intelligence• ““Common sense” as well as social or Common sense” as well as social or
emotional intelligence; “street smarts”emotional intelligence; “street smarts” Creative IntelligenceCreative Intelligence
Ability to achieve success in life and work; requires a balance among:
Analytical IntelligenceAnalytical Intelligence
Traditional aspects of intelligence Traditional aspects of intelligence measured by standard testsmeasured by standard tests• Verbal and numerical ability, memory, Verbal and numerical ability, memory,
processing speedprocessing speed Predicts job performance in nearly all jobsPredicts job performance in nearly all jobs
• More important as jobs become more complexMore important as jobs become more complex• Probably due to:Probably due to:
Broader knowledge baseBroader knowledge base Ability to learn on the job more quicklyAbility to learn on the job more quickly Ability to problem-solveAbility to problem-solve
Practical IntelligencePractical Intelligence Ability to deal with everyday life challengesAbility to deal with everyday life challenges
• E.g., computing lowest price/unit when grocery E.g., computing lowest price/unit when grocery shopping, or interpreting a bus scheduleshopping, or interpreting a bus schedule
Social/emotional intelligenceSocial/emotional intelligence• Ability to understand people and relationshipsAbility to understand people and relationships• Impression management skillsImpression management skills• Ability to persuade othersAbility to persuade others
Adaptability to different social situationsAdaptability to different social situations Shown to add to predictive power of Shown to add to predictive power of
analytical intelligence for sales and execsanalytical intelligence for sales and execs May be culture-specificMay be culture-specific
Creative IntelligenceCreative Intelligence
Relies on a base of knowledge + Relies on a base of knowledge + ability to apply it to new situations or ability to apply it to new situations or problems problems • Ability to generate novel high-quality Ability to generate novel high-quality
ideasideas• Ability to recognize opportunities or Ability to recognize opportunities or
define challenges in new waysdefine challenges in new ways• Ability to adjust to new situationsAbility to adjust to new situations
Measuring IntelligenceMeasuring Intelligence
Wide variety of measures of analytical Wide variety of measures of analytical intelligence/general mental abilityintelligence/general mental ability• Wonderlic Personnel Test is most popularWonderlic Personnel Test is most popular
Sternberg Multidimensional Abilities Test Sternberg Multidimensional Abilities Test measures all 3 intelligencesmeasures all 3 intelligences
Sternberg and Wagner’s test of Practical Sternberg and Wagner’s test of Practical Managerial Intelligence: Managerial Intelligence: • ability to write effective memos ability to write effective memos • ability to motivate people ability to motivate people • knowledge of when to delegate knowledge of when to delegate • ability to “read” people ability to “read” people
Lingering Controversies about Lingering Controversies about IQIQ
Is intelligence testing fair?Is intelligence testing fair? Is intelligence—even when measured Is intelligence—even when measured
more broadly—sufficient?more broadly—sufficient? Do you want “bright” or “well Do you want “bright” or “well
trained” employees?trained” employees?
Fairness and Intelligence Fairness and Intelligence Testing: A DilemmaTesting: A Dilemma
CA predicts performance in a wide CA predicts performance in a wide variety of jobsvariety of jobs• Measures knowledgeMeasures knowledge• Measures ability to learn, reason, and Measures ability to learn, reason, and
solve new problemssolve new problems• Few alternative predictors with Few alternative predictors with
comparable utility for entry-level jobscomparable utility for entry-level jobs But are they “fair” to all groups?But are they “fair” to all groups?
True Validities of Predictors For Entry-Level Jobs
Ability Test Battery .53 Interview (structured) .44 Job Tryout .44 Biographical Inventory .37 Reference Checks .26 Personality Inventories ~.23 Experience Ratings .18 Interview (unstructured) .14 Academic Achievement .11 Interest Inventories .10
For Promotions Work Sample .54 Ability Test Battery .53 Peer Ratings .49 Experience Ratings .49 Job Knowledge Test .48 Assessment Center .43
1. Differences between groups’ average test 1. Differences between groups’ average test scoresscores• ~1 SD difference for African-Americans~1 SD difference for African-Americans• ~.75 SD for Latinos~.75 SD for Latinos
But what do these differences mean?But what do these differences mean?
Definitions of Test “Fairness”Definitions of Test “Fairness”
• Groups differ in ability?• Test discriminates against one group?
Definitions of Test “Fairness”Definitions of Test “Fairness”
2. 2. Differential ValidityDifferential Validity: Group : Group differences in differences in validityvalidity of test of test
Definitions of Test “Fairness”Definitions of Test “Fairness”
2. 2. Differential ValidityDifferential Validity: Group : Group differences in validity of testdifferences in validity of test
White
Black
Test Scores
JobPerf
Definitions of Test “Fairness”Definitions of Test “Fairness”
3. Test Bias3. Test Bias: Group differences in : Group differences in predicted job performance with same predicted job performance with same cutoff scorecutoff score
Definitions of Test “Fairness”Definitions of Test “Fairness”
3. Test Bias3. Test Bias: Group differences in : Group differences in predicted job performance with same predicted job performance with same cutoff scorecutoff score
Black
Test Scores
JobPerf
White
Definitions of Test “Fairness”Definitions of Test “Fairness”
3. Test Bias3. Test Bias: Group differences in : Group differences in predicted job performance with same predicted job performance with same cutoff scorecutoff score
Black
Test Scores
JobPerf
White
Typical CA Test SituationTypical CA Test Situation
Adverse ImpactAdverse Impact No differential validityNo differential validity No test biasNo test bias
White
Black
Test Scores
JobPerf
Typical CA Test SituationTypical CA Test Situation
Adverse ImpactAdverse Impact No differential validityNo differential validity No test biasNo test bias
White
Black
Test Scores
JobPerf
Minimizing Adverse ImpactMinimizing Adverse Impact
Recruit high quality applicantsRecruit high quality applicants Use tests only when appropriateUse tests only when appropriate Use tests as one part of processUse tests as one part of process Use to eliminate unqualified (rather Use to eliminate unqualified (rather
than to differentiate among most than to differentiate among most qualified)qualified)
Be sure test is valid (job-related)!!Be sure test is valid (job-related)!!
Lingering Controversies about Lingering Controversies about IQIQ
Is intelligence testing fair?Is intelligence testing fair? Is intelligence—even when measured Is intelligence—even when measured
more broadly—sufficient?more broadly—sufficient? Do you want “bright” or “well Do you want “bright” or “well
trained” employees?trained” employees?
Why Intelligent People FailWhy Intelligent People Fail
Lack of motivation Lack of motivation Lack of impulse control Lack of impulse control Lack of perseverance Lack of perseverance Fear of failure Fear of failure Procrastination Procrastination Inability to delay gratification Inability to delay gratification Too little/too much self-confidence Too little/too much self-confidence
Source: Sternberg Source: Sternberg (1986)(1986)
Lingering Controversies about Lingering Controversies about IQIQ
Is intelligence testing fair?Is intelligence testing fair? Is intelligence—even when measured Is intelligence—even when measured
more broadly—sufficient?more broadly—sufficient? Do you want “bright” or “well Do you want “bright” or “well
trained” employees?trained” employees?
Intelligence or Training?Intelligence or Training?
Is being smart Is being smart a pre-requisitea pre-requisite for learning? for learning? Is being smart Is being smart sufficientsufficient for learning? for learning?
InvasivenessInvasiveness
Employer right to know vs. right to Employer right to know vs. right to privacyprivacy
Over-emphasis on individual itemsOver-emphasis on individual items Reducing invasiveness:Reducing invasiveness:
• Use only for job-related traitsUse only for job-related traits• Use employment (not clinical) measuresUse employment (not clinical) measures• Use as part of overall systemUse as part of overall system• Explain purpose as matching not Explain purpose as matching not
screeningscreening
Extreme Items from MMPI
I have never indulged in any unusual sex practices.
I have never been in trouble because of my sex behavior.
My soul sometimes leaves my body.
Evil spirits possess me at times.
I sometimes feel like I’m going to pieces.
Often I feel as if there were a tight band around my head.
I have had no difficulty starting or holding my bowel movements.
Sample Items Modeled After Those on NEO-PI
I am generally not a worrier.
I would rather cooperate than compete.
I don’t mind bragging about what I’ve accomplished.
I’m pretty set in my ways of doing things.
I try to keep all my thoughts realistic and to avoid flights of fancy.
I have no sympathy for “street people.”
I prefer jobs that allow me to work alone without being interrupted by others.
I am a person who always gets the job done.
Is Personality Job-Relevant?Is Personality Job-Relevant?
The “Big Five” Personality TraitsThe “Big Five” Personality Traits• ConscientiousnessConscientiousness• Emotional StabilityEmotional Stability• ExtraversionExtraversion• AgreeablenessAgreeableness• <Openness to Experience><Openness to Experience>
Others may apply in particular jobs Others may apply in particular jobs • e.g., customer service, safety, attention e.g., customer service, safety, attention
to detailto detail
Do Personality Tests Predict?Do Personality Tests Predict?
Is the situation Is the situation weakweak or or powerfulpowerful?? Is the measure valid?Is the measure valid? Validities typically in low .20s (when Validities typically in low .20s (when
based on job analysis)based on job analysis)
Faking & Personality TestsFaking & Personality Tests
Applicants often distort their answers Applicants often distort their answers to look goodto look good• Some applicants more than othersSome applicants more than others• This can affect who is hiredThis can affect who is hired
Reducing faking:Reducing faking:• Use faking (lie) scales Use faking (lie) scales • Reduce motivation to fakeReduce motivation to fake• Use less transparent questionsUse less transparent questions
Social Desirability Scale Items
I sometimes drive faster than the speed limit.
My table manners are not quite as good at home as when I’m out in company.
I gossip a little at times.
Sometimes I vote for people I know very little about.
I do not like everyone I know.
I sometimes tell lies if I have to.
Fairness of Personality TestsFairness of Personality Tests
Little evidence of racial Little evidence of racial discriminationdiscrimination
Men and women differ on many Men and women differ on many scalesscales• but no evidence of “discrimination”but no evidence of “discrimination”
Watch out for adverse impact due to:Watch out for adverse impact due to:• religious beliefsreligious beliefs• disabilitydisability
Ways to Assess PersonalityWays to Assess Personality
Paper & Pencil InventoriesPaper & Pencil Inventories Individual (clinical) assessmentIndividual (clinical) assessment During interviewsDuring interviews
Ways to Assess PersonalityWays to Assess Personality
Correlation Between Self- and Interviewer-Ratingsr
Extraversion 0.27Conscientiousness 0.22Openness to Experience 0.11Agreeableness -0.29Emotional Stability -0.30
Ways to Assess PersonalityWays to Assess Personality
Paper & Pencil InventoriesPaper & Pencil Inventories Individual (clinical) assessmentIndividual (clinical) assessment During interviewsDuring interviews Reports from othersReports from others Behavioral observationBehavioral observation
Tools for Successful Tools for Successful SelectionSelection
Assessing Deviance:Assessing Deviance:
Drug TestingDrug Testing
Integrity (Honesty) TestingIntegrity (Honesty) Testing
What is “Job Performance”?What is “Job Performance”?
In-Role Behavior
Extra-Role
Behavior
Counter-Productive Behavior
Risk Factors
General RationaleGeneral Rationale
Employee deviance is a costly Employee deviance is a costly problemproblem• $6 to 200 billion annual cost $6 to 200 billion annual cost • 33-75% of employees 33-75% of employees
It’s said that 30-50% of business It’s said that 30-50% of business failures are due to theftfailures are due to theft
Drug/alcohol abuse results in $197B Drug/alcohol abuse results in $197B in productivity costsin productivity costs
How Do You Screen out How Do You Screen out Deviant Employees?Deviant Employees?
Some are based on past behaviorSome are based on past behavior• analysis of drug metabolites in urine; analysis of drug metabolites in urine;
past theft; reference and background past theft; reference and background checkschecks
Others focus on likely future behaviorOthers focus on likely future behavior• attitudes toward theft, drug use; attitudes toward theft, drug use;
personalitypersonality Only a few focus on current behaviorOnly a few focus on current behavior
• impairment testingimpairment testing
Physiological Drug TestsPhysiological Drug Tests
UrinalysisUrinalysis• Quick screeningQuick screening• Confirmatory: Gas Confirmatory: Gas
Chromatography/Mass Spectrometry Chromatography/Mass Spectrometry (GC/MS)(GC/MS)
Hair analysisHair analysis Blood analysisBlood analysis Breath analysisBreath analysis
What’s Measured (& for how What’s Measured (& for how long?)long?)
The “NIDA 5”:The “NIDA 5”:• CannabisCannabis• CocaineCocaine• AmphetaminesAmphetamines• PCPPCP• OpiatesOpiates
What’s Measured (& for how What’s Measured (& for how long?)long?)
The “NIDA 5”:The “NIDA 5”:• CannabisCannabis 2 – 90 days2 – 90 days• CocaineCocaine 1 – 2 days1 – 2 days• AmphetaminesAmphetamines .5 – 2 days.5 – 2 days• PCPPCP 30 – 90 days30 – 90 days• OpiatesOpiates 2 – 4 days2 – 4 days
Other Forms of Drug TestsOther Forms of Drug Tests
Paper-and-pencil integrity testsPaper-and-pencil integrity tests Impairment testingImpairment testing
• based on person’s own “baseline” based on person’s own “baseline” performance on a computer taskperformance on a computer task
Types of Drug Testing Types of Drug Testing ProgramsPrograms
Pre-employmentPre-employment ““For cause” testingFor cause” testing
• mandatory post-accidentmandatory post-accident• based on supervisor’s observationsbased on supervisor’s observations
Fitness for duty Fitness for duty • daily (impairment testing)daily (impairment testing)• post-rehabilitationpost-rehabilitation
RandomRandom
Types of Honesty/Integrity TestsTypes of Honesty/Integrity Tests
““Clear Purpose” testsClear Purpose” tests ““Disguised Purpose” testsDisguised Purpose” tests
• Attitudes about devianceAttitudes about deviance• PersonalityPersonality
Polygraph/Voice Stress AnalyzerPolygraph/Voice Stress Analyzer
Clear Purpose TestClear Purpose Test - Honesty/Morality Items- Honesty/Morality Items
If you received an extra $1.00 in If you received an extra $1.00 in change when you purchased change when you purchased groceries, would you keep it? groceries, would you keep it?
If you saw a co-worker stealing, would If you saw a co-worker stealing, would you turn her in? you turn her in?
How many people have you known How many people have you known who have stolen valuables at work? who have stolen valuables at work?
Most people cheat on their income Most people cheat on their income tax. tax.
Clear Purpose Test:Clear Purpose Test: - Drug-Related Items - Drug-Related Items
I have never been a heavy drinker. I have never been a heavy drinker. On occasion I have used On occasion I have used
alcohol/drugs excessively alcohol/drugs excessively If I thought I could get away with it, If I thought I could get away with it,
I'd probably try cocaineI'd probably try cocaine Have you ever missed work because Have you ever missed work because
you were too high/drunk?you were too high/drunk?
Types of Honesty/Integrity TestsTypes of Honesty/Integrity Tests
““Clear Purpose” testsClear Purpose” tests ““Disguised Purpose” testsDisguised Purpose” tests
• Attitudes about devianceAttitudes about deviance• PersonalityPersonality
Polygraph/Voice Stress AnalyzerPolygraph/Voice Stress Analyzer
Disguised Purpose TestDisguised Purpose Test - - Drug Attitude ItemsDrug Attitude Items
A drug addict who steals to support A drug addict who steals to support his habit should get the same his habit should get the same penalty as a person who steals for penalty as a person who steals for profit. profit.
Employers expect a certain amount Employers expect a certain amount of drug and alcohol use among their of drug and alcohol use among their workers. workers.
How many people have you known How many people have you known who drink or do drugs while working? who drink or do drugs while working?
Disguised Purpose Test:Disguised Purpose Test: - - Personality ItemsPersonality Items
I enjoy making people feel better.I enjoy making people feel better. I don't care if others like the things I I don't care if others like the things I
do.do. I shouldn't do many of the things I do.I shouldn't do many of the things I do. I have never been in trouble with the I have never been in trouble with the
law.law. Sometimes I enjoy going against the Sometimes I enjoy going against the
rules.rules.
Types of Honesty/Integrity TestsTypes of Honesty/Integrity Tests
““Clear Purpose” testsClear Purpose” tests ““Disguised Purpose” testsDisguised Purpose” tests
• Attitudes about devianceAttitudes about deviance• PersonalityPersonality
Polygraph/Voice Stress AnalyzerPolygraph/Voice Stress Analyzer
Drug & Integrity Testing IssuesDrug & Integrity Testing Issues
LegalityLegality Accuracy/Job relevanceAccuracy/Job relevance Faking (integrity tests)Faking (integrity tests) Consequences of failingConsequences of failing Negative reactions (invasion of Negative reactions (invasion of
privacy/fairness)privacy/fairness)
Integrity Testing: A CautionIntegrity Testing: A Caution Integrity tests assume cause of Integrity tests assume cause of
deviance is bad employeesdeviance is bad employees But the situation may also be a factorBut the situation may also be a factor
• role models, group norms may “push” role models, group norms may “push” people towards deviance people towards deviance
• dissatisfaction may “pull” towards dissatisfaction may “pull” towards deviancedeviance
Integrity tests don’t replace good Integrity tests don’t replace good managementmanagement
Tools for Successful Tools for Successful SelectionSelection
Employment Testing: Employment Testing:
Performance Tests & Work Performance Tests & Work SamplesSamples
Signs vs SamplesSigns vs Samples
SignsSigns: indirect indicators of : indirect indicators of performance potentialperformance potential
SamplesSamples: direct indicators of : direct indicators of performance potentialperformance potential
Sign-Sample ContinuumSign-Sample Continuum
Probationary employmentProbationary employment ApprenticeshipsApprenticeships InternshipsInternships Mock-upsMock-ups Assessment centersAssessment centers Role-playing interviewsRole-playing interviews
Paper/pencil testsPaper/pencil tests
Sample
Sign
Pros & ConsPros & Cons
High validityHigh validity Low adverse impactLow adverse impact High face validityHigh face validity
Advantages
• More time consuming• Direct costs
Disadvantages
Step 1: Define Performance Step 1: Define Performance
Which performance dimensions?Which performance dimensions?• Refer to Performance Attributes MatrixRefer to Performance Attributes Matrix• Don’t focus only on easily simulated Don’t focus only on easily simulated
dimensionsdimensions
Performance Attributes MatrixPerformance Attributes Matrix
Attribute
App.
Blank Interview
Reference
Checks
Work
Sample
Knowledge
Test
Personality
Inventory
Ability to workaccurately
X X X X
Ability to workwell with customers
X X X X
Work-specificknowledge
X X X
Computer-operating skill
X X
Conscientiousness X X X
Step 1: Define Performance Step 1: Define Performance
Which performance dimensions?Which performance dimensions?• Refer to Performance Attributes MatrixRefer to Performance Attributes Matrix• Don’t focus only on easily simulated Don’t focus only on easily simulated
dimensionsdimensions Determine reasonable standards for Determine reasonable standards for
the test/samplethe test/sample What type of validation would you What type of validation would you
use?use?
Step 2: Creating SampleStep 2: Creating Sample
Key principlesKey principles• SpecificitySpecificity• Comprehensiveness (vs Narrowness)Comprehensiveness (vs Narrowness)• FidelityFidelity• PracticalityPracticality
Use Performance Attributes Matrix to:Use Performance Attributes Matrix to:• Ensure all KSAs are included in planEnsure all KSAs are included in plan• Weight all KSAs appropriatelyWeight all KSAs appropriately
Example: Example: Marketing SpecialistMarketing Specialist
Performing market analysisPerforming market analysis• Have candidates perform a market Have candidates perform a market
analysis for a particular product/marketanalysis for a particular product/market Developing appropriate marketing Developing appropriate marketing
materialsmaterials• Critique current materialsCritique current materials
Developing a marketing planDeveloping a marketing plan• Develop part of a planDevelop part of a plan