how predictive is your recruitment method? · evidence based leadership sacs consulting is carbon...

Evidence Based LeadershipSACS Consulting is carbon neutral certified.

How predictive is your recruitment method?

Andrew MartyManaging Director

SACS Consulting


SACS Consulting

• Human Resource Management Consulting Firm– Executive Search and Selection– Human Resource Management Consulting


Turning latest research into best practice


How predictive is your recruitment method?

Andrew MartyManaging Director


Objectives

• To review the “league ladder” of recruitment methods• To examine the most predictive methods in more detail• To suggest an optimum approach.


Evidence

• The findings in this information come from research, some time honoured and some very recent, particularly that relating to personality measures. If you want references I am happy to provide them.


Recruitment methods are measures

• They should allow us to rank and rate candidates• They should be valid and reliable• The most powerful (and difficult to achieve) form of validity

is predictive validity. This is the one you should care about.


League Ladder of Recruitment Predictiveness

Measures Validity (r)

Cognitive ability and Integrity 0.65

Cognitive ability and Structured Interviews 0.63

Cognitive ability and work sample 0.60

Work sample tests 0.54

Cognitive ability 0.51

Structured interviews 0.51

Job knowledge tests 0.48

Integrity Tests 0.41

Personality tests – big 5 with a focus on Neuroticism , Conscientiousness and Agreeableness 0.40

Assessment Centres 0.37

Unstructured interviews 0.35

Biodata 0.35

Conscientiousness 0.31

References 0.26

Years job experience 0.18

Interests 0.1

Years of Education 0.1

Graphology 0.02

Age -0.01

Source: Modified from Robertson, I.T & Smith, M. (2001)

Personnel Selection Journal of Occupational and Organisational Psychology (2001), (74), 441-472.


Things which do not predict

• Graphology – sad for the French• Age • Years of education • Years of work


What does work?

• Testing is better than interviewing – significantly better – in the top 5 methods all involve some form of testing. I know we find this hard to believe, but this finding has been replicated for decades.– Work sample testing– Cognitive ability– Integrity testing– Job knowledge tests


Cognitive ability testing

• Assess your candidate’s intelligence. Very predictive - .51• Verbal reasoning

– The capacity to use words effectively– People higher on this measure are better able to express

ideas easily, and to influence others– Good predictor of communication success


Cognitive ability testing

• Numerical reasoning– The capacity to use and understand numbers– Large impact on ease of dealing with finances, performance

measures, statistics, etc.• Abstract reasoning

– The capacity to solve abstract problems which do not involve numbers or words

– Very important ability in areas such as strategy, planning, problem solving


How are they accessed?

• Easily accessed tests– ACER ML/AL– ACER MQ/AQ– ACER APTS

• Cognitive ability tests partner well with other methods because they assess elements which other assessment methods do not


Integrity tests

• Assess the person’s values as they relate to integrity and ethical values

• Surprisingly predictive– .41 by themselves and when coupled with cognitive ability

tests .65. They add together well because they are assessing such different things

– They get more predictive over the years. Bizarre but true!• Aim to eliminate candidates with negative characteristics –

up to 90% in some studies • Targeting candidates with positive attitudes

– i.e. organisational citizenship behaviours


Examples

• EG Reid Report– Integrity– Attitude– Social Behaviour – Safety/substance abuse– Personal achievements


Integrity tests

• Theorised to be so predictive because they relate closely to a whole positive value set which indicates a good employee

• Extremely rarely used in Australia, apart from the forces• Should be used more!


Interviews

• Unstructured = .35; Structured = .51• The better the structure the more predictive• The vital elements of the most predictive structure are:

– A competency analysis for each job– Behavioural interview questions targeting the key

competencies– A scoring structure for each of the key competencies.


Competency analysis

• A competency analysis is a professional version of a person specification

• Much more valuable than a job description


Competency anlaysis

• What are the Key Result Areas (KRA) the job needs to deliver?

• Competencies necessary to deliver the KRAs– Skills– Experience/qualifications– Style, values, approach

• Look totally laterally for candidates who have these competencies


Behavioural interview questions

• Ask about Critical Incidents• Please Give an Example (PGAE questions)

– Eg: Please give an example where you have been successful in recruiting high calibre people, improving the outcomes of a work unit

• Way more predictive then “tell us about yourself” or “what would you do in the following situation” questioning


Worksample testing

• Very predictive– .54 in its own right

• Take real examples of work to be completed and put them in a form which can be undertaken in a pre-employment format. For example:– Analyse a set of financials from the organisation – rate their

analysis the same way you would for an employee– Ask them to write a report similar to what they would do on the job– Solve a problem from the workplace, and make a presentation,

either by themselves, or in a group– Internships


Worksample testing is not….

• Role playing – picks out good actors rather than good employees

• In-basket exercises– Very helpful in staff development, but not predictive of

employment success. They also tend to pick out good strategists rather than implementers.


Job knowledge tests

• Tests designed to assess the candidate’s subject matter knowledge - .48

• The more specifically tailored the more predictive• Work best when knowledge of subject matter is critical to a

role – compliance, technical, design, product knowledge, etc.

• There are standard job knowledge tests for certain standard jobs – some are created by large employers.


Personality tests

• A recent entrant into the top half of the league ladder - .40• Big 5 model of personality is more predictive than anything

previous– Neuroticism *– Extraversion – Openness– Agreeableness*– Conscientiousness* - also has some integrity testing

elements to it


Neuroticism and Conscientiousness

• If you assess any personality measures you should assess these

• Conscientiousness gives .31 in its own right. Adding multiple measures of conscientiousness adds validity. Up to .45

• Neuroticism and agreeableness are excellent predictors of negative outcomes. Now estimated to be .3+

• Irrespective of what the job is, the research suggests that you should hire people with low neuroticism scores and high agreeableness and conscientiousness scores.


NEO personality measure

Neuroticism– Anxiety– Angry hostility– Depression– Self consciousness– Impulsiveness – Vulnerability

Conscientiousness– Competence– Order– Dutifulness– Achievement striving– Self-discipline– Deliberation

Agreeableness– Trust – Straightforwardness – Altruism– Compliance– Tender mindedness– Modesty


Style of Well-BeingVertical Axis: Neuroticism (= 41 T )

Horizontal Axis: Extraversion (= 68 T )

20 30 40 50 60 70 80

20

30

40

50

60

70

80

HighNeuroticism

LowNeuroticism

HighExtraversion

LowExtraversion

N+E- N+E+

N-E- N-E+

Gloomy Pessimists Overly Emotional

Low-keyed Upbeat Optimists

These people face a dark and dreary life. There is littlethat cheers them and much that causes anguish anddistress. Especially under stressfulcircumstances, they may succumb toperiods of clinical depression, andeven when they are functioningnormally, they often find life hardand joyless.

These people experience both positive and negativeemotions fully and may swing rapidly from one

mood to another. Their interpersonalinteractions may be tumultuous because

they are so easily carried away by theirfeelings. They may show features of

the Histrionic PersonalityDisorder, but they may also feel

that their lives are full ofexcitement.

Neither good news nor bad hasmuch effect on these people; theymaintain a stoic indifference to eventsthat would frighten or delight others.Their interpersonal relationships may sufferbecause other people find them to be "cold fish."Their emotional experience of life is bland.

These people are usually cheerfulbecause they are not unduly

troubled by problems, and they have akeen appreciation for life's pleasures.

When faced with frustration ordisappointment, they may become angry or sad, but

they quickly put these feelings behind them. They preferto concentrate on the future, which they view with eageranticipation. They enjoy life.


Personality testing continued

• Beware any personality measures which are proprietary – we have no idea how good they are

• We use only peer reviewed measures• We are so committed to this we are making it part of our

standard recruitment method.


Assessment Centres

• Evidence for their predictiveness varies very widely• For research purposes, there has been little distinction

between different types of assessment centres• The best seem to be very predictive, the worst only

moderately predictive• Average of .37


Pluses and minuses

• Assesses characteristics which can be assessed only in a social context

• Assesses very different material from any other recruitment tool

• Powerful in assessing values, style etc

• Can be very economical for prequalifying candidates – low cost per assessment

• Allows for multiple assessors

• Can only be used where confidentiality issues do not matter

• People feel they can be applied only to junior hires

• Can be expensive to set up in the first instance

• Need to be well designed – appear to be very sensitive to poor planning – lack of competency analysis, etc.


A/C Centre design features

• Should be tailored to each application – if it is off-the-shelf, it will be less predictive. Most assessment centres are standard. Virtually all outsourced ones are

• Should be based on competency analysis and behavioural principles

• Should involve work sampling elements• Role playing is less valuable than work problem solving• Should be used as a pre-selection device for structured

interviews and psych testing


Typical components

• Introductions – all assessors and then participants• Group discussion questions, based around the

competencies sought• Problem solving or other work sampling activities, often

requiring participants to present results to the larger group• Question and answer session on the job being sought –

assessing quality of questions and responses of candidates.


Traditional method of assessment

Construct Rating out of 10

1. Commitment to customer service

2. Problem solving ability

3. Oral communication skills

4. Etc.

TOTAL


More predictive method of assessment

Activity Rating out of 10

1. Introduction

2. Group Discussion

3. Problem solving activity number 1

4. Etc.

TOTAL


Biodata

• Once you have identified the competencies required to achieve the KRAs, consider what kind of work experience indicates these competencies

• Develop a simple scoring method for backgrounds. As simple as A B C!

• Predictiveness of .35


Biodata assessment example

Activity Rating out of 10

1. Evidence of profit centre management experience

2. Evidence of having managed growth

3. Has led teams in multiple locations

4. Etc.

TOTAL


Toward an optimum method

• Start with a competency analysis for the role – the only way you will end up with a truly structured interview. High performance modelling is best, but single case competency analysis is fine.


High Performance Modelling

IDENTIFICATIONOF HIGH

PERFORMERS-“EXEMPLARS”

AND THEIR KRAs

HIGH PERFORMANCE MODEL

Use the psychological tests and their scale scores which pick

out the high performers.

List the competencies which all your high performers have in common. Write behavioural

interview questions and a simple scoring system for each.

Behavioural Interviews What skills, knowledge, values and attitudes do they have in common?

Psychological testingWhat psychological characteristics

do they have in common?


An optimum assessment method

• Evaluate candidates through biodata scoring. Invite only A candidates to the next stage

• Use an assessment centre if possible• Use structured interviews involving behavioural interview

questions and competency ratings• Psych test the candidates who make it through

– Cognitive ability – verbal, numerical, abstract reasoning– Big 5 personality measure with an emphasis on

conscientiousness and neuroticism– Integrity testing


Optimum assessment method continued

• Those who are still live are told they are at the last stage of assessment and are given the opportunity to participate in a work sampling activity. Choose a “sheep from goat” activity out of the work place and rate their ability to do it. Can be done even at very senior levels.

• Undertake behavioural reference checking – assessing the previously identified competencies. Get your last .26 worth of value!


Outcome?

If you do these things you should reliably achieve predictiveness above .70. The predictiveness of most

organisations’ methods is around .35-.40. This is what’s known as a competitive edge.


Summary and conclusions

• Recruitment without psych testing is unsupportably risky• We now have better information than ever as to what kind of

psych testing works• Beware proprietary instruments which have not been peer

reviewed• Undertake competency analysis and behavioural interviewing

processes – they make a huge difference• Build work sampling into your recruitment methods. A little

creativity can add enormous benefit • Consider integrity testing – infrequently used, but highly

predictive.

Major SponsorsConference Partner Principal Sponsor

National Corporate Partner

Media Supporters

National Corporate Partner

National Principal Sponsorp p

FIA Corporate Supporter

Awards Sponsors

how predictive is your recruitment method? · evidence based leadership sacs consulting is carbon...

Documents