OCCUPATIONAL PERSONALITYQUESTIONNAIRES

(OPQ)

LDpe/SHL Pre-Course Guide

SHL’s OPQ measurement system is used as the basic measurement for LDpe’s Leadership Assessment.

This reader contains your delegate information, and pre-course work concerning the OPQ measurement system.PLEASE READ IT CAREFULLY ANDCOMPLETE THE EXERCISES SHOWN.

February 2008Updated by LDpe

1

CONTENTS Page

Number

1. DELEGATE INFORMATION 42. PRE COURSE READING AND EXERCISES 93. STATISTICS REVISION GUIDE 594. ANSWERS TO THE EXERCISES 66

2

SECTION 1

Delegate Information about the OPQ training

3

1. DELEGATE INFORMATION

1.1 Pre-Course Work

In order to fully benefit from attending the course, you will need to do some preparatory work in advance of attending. This pre-course work is likely to take several hours - we strongly recommend that you do not attempt to complete this all in one sitting. Full details of the pre-work are provided in Section Two “Pre-Course Work”.

1.2 Course Materials

Please bring with you to the course all of the materials sent to you. All other materials required for the course will be provided. You should, however, bring a calculator that has a square root function.

1.3 Questionnaires to complete before joining the training program

There is a questionnaire to complete before coming to the course:If you have not already done so, shortly you will receive an e-mail inviting you to complete the OPQ32 ipsative version on-line. The email will contain a user name and password together with instructions on how to access the website to complete the questionnaire. Completion of the questionnaire will take around 40 minutes and should be done in one sitting. The outcome will be available at the start of the course.

1.4 Introduction to Personality Measurement

Section Two of this pre-course guide is an introduction to personality and its assessment. You will need to read this material, and complete the exercises included before attending the course. These exercises ask you to refer to material within the pre-course guide and also to consider your own experiences. This pre-course work on this OPQ reader is likely to take in total about 2-3 hours. The Course Director will assume that you are familiar with this material and will encourage comment/discussion and further questions.

1.5 Review of Statistics

This course assumes understanding of the basic statistical concepts underlying norms, reliability and validity. Section Three of this booklet is a Statistics Revision Guide. This is designed to remind you of some basic statistical concepts. Work through the questions and check your answers against those provided at the back of the booklet.

4

1.6 OPQ TRAINING OBJECTIVES

The OPQ-training will enable delegates to:

Gain a critical appreciation of personality theories and assessment methods

Achieve proficiency in administering, interpreting and feeding back the OPQ

Understand how to introduce and apply the OPQ in relation to LDpe’s Leadership Assessment

Become familiar with the latest research findings in personality assessment

Qualify to explain the OPQ personality profile as the basis of the Leadership Assessment Toolbox

5

SECTION 2

Pre course Reading and Exercises

6

Section 2 Table of Contents

1. What is Personality?

2. Assumptions About Personality

3. Methods of Assessing Personality

4. Overview of the OPQ Family

5. The Development of the OPQ

6. Applications of the OPQ

7. Administering the OPQ

8. Reliability

7

SECTION 2: MODULE 1 - LEARNING OBJECTIVES:

To understand what is meant by “Personality”.

To gain an appreciation of how personality differs from other aspects of human behaviour.

To consider the relevant of personality to performance at work.

To look at approaches to considering personality.

8

2. PRE COURSE READING AND EXERCISES

2.1 Introduction to Personality Measurement

2.1.1 What is Personality?

Research suggests that although we are able to adapt our behaviour to particular circumstances, there is nevertheless considerable consistency in our behaviour, across situations and over time. It is these consistencies in behaviour which constitute the essence of most definitions of personality.

2.1.2 The OPQ Definition of Personality

We suggest that personality is best defined as “a person’s typical or preferred way of behaving, thinking and feeling.”

This recognises that behaviour is always to some extent determined by current environment and circumstances. The emphasis, however, is upon those relatively stable and enduring characteristics which determine differences between individuals in their typical manner of relating to other people, approaching tasks and responding to situations generally.

2.1.3. Other elements that affect the way we behave

MOTIVATION What drives an individual and directs and sustains their behaviour.

ATTITUDES“A learned orientation or disposition towards an object or situation which provides a tendency to respond either favourably or unfavourably to the object or situation.” (Rokeach, 1968)

BELIEFS Patterns of behaviour which, through a process of socialisation, are held in high regard by members of a particular society.

VALUES “A belief by which someone acts by preference.” (Allport, 1935)

INTERESTS Something which arouses attention and curiosity.

ABILITY Qualities, competence and talent which enable someone to perform a task

Exercise 1 – Look at the following quote. After each statement indicate whether you feel it is about personality, motivation, values, attitude, interest or ability. Each item should be used only once.

“I’ve always thought I’d like to work in sales (_____________), particularly in this organisation which regards the sales force highly, (____________). I’m a bit worried that I’d argue too much with difficult customers (_____________) and anyway, I’m bound to make mistakes in reporting my sales figures (______________) but then I’ve never regarded numbers as that important (______________). It’s a shame, because the feeling of making a sale would spur me on.” (______________)

9

2.1.3 Is Personality Relevant at Work?

Results from a large number of job analysis projects indicate that up to 70% of attributes associated with success at work are aspects of personality rather than ability. To put that into some perspective, consider the following competency criteria for senior manager assessment used in one airline. Note how all of these competencies have some relationship with personality.

Table 1: Competencies for Senior Management in an Airline

Influence/Leadership Able to take charge and control a group towards an objective. Makes their presence felt in groups. Motivates subordinates, delegates effectively and monitors performance.

Communication Skills Able to comprehend and express ideas accurately and persuasively both orally and in writing.

Organising & Planning Able to think ahead, prioritise and organise work. Is able to structure activities and meet targets within appropriate budgets.

Motivation/Energy Shows energy, drive and enthusiasm. Maintains a high output. Proactive and ambitious.

Creativity Generates and is receptive to new ideas. Seeks to innovate rather than necessarily accept the established solution.

Analytical Reasons objectively and critically with both verbal and numerical problems. Comprehends and processes information at a high level of complexity.

Empathy Able to understand the strengths, weaknesses, views and feelings of others. Supports others and is a good team member.

Emotional Maturity Responds well to criticism. Frank and open. Able to take pressure.

Decision Making Having the confidence to take own decisions. Being prepared to take a balanced risk and to be held accountable for the outcome.

Commercial Awareness An interest in, and understanding of, the financial/profit implications of actions taken. A hard-headed concern for the ‘bottom line’.

10

2.1.5 Theoretical Approaches to Personality

The IMPLICIT Approach - Most people probably feel they understand a little about human nature, and some may believe they understand other people very well indeed. People who feel this way needn’t necessarily have any specific theory about why we behave the way we do. They simply feel they “know how people behave,” and they may or may not be right in this assumption. Certainly some managers, human resource specialists - even sports coaches - can have a better instinctive “feel” for what makes people tick than the average psychologist! Implicit models of personality are never defined, stated or written down, but people act on them all the time, sometimes effectively, sometimes not.

The EXPLICIT Approach – Explicit models of personality are defined, and are likely to be based either on empirical research or on some psychological (perhaps clinical) theory of personality. The research may be flawed, or the theory a peculiar one, so an explicit model isn’t necessarily a correct one An explicit model is clearly defined and can be investigated by others to see if it stands up or if it should be modified or dropped altogether.

There have been many explicit models of personality over the years. The following table outlines some of the more well known ones.

Exercise 2 – Look at the statements below and, alongside each one, indicate whether you feel it suggests an implicit or an explicit approach to personality (N.B. an explicit theory is not necessarily a correct one!)

1. Ginger haired people are always miserable2. You can tell you’re a Tory just by looking at you3. Anyone who works as an accountant is bound to be boring4. I can spot a good candidate as soon as they walk through the door

5. Now write down an implicit view of personality that you hold:

6. Now write down an explicit view of personality with which you agree:

11

7. And write down an explicit view of personality with which you disagree:

12

2.1.6 Overview of Theoretical Approaches to Personality

Psychoanalytic Approach

Psychometric Approach

Social Learning Approach

Humanistic Approach

MainProponents

Sigmund FreudCarl Jung

Hans EysenckRaymond Cattell

Albert BanduraRobert Walters

Abraham Maslow

Carl RogersOverview ofTheory

Emphasises the power of the subconscious, with 3 main components to personality: The ID is the oldest and most obscure part of personality. It is the primary source of psychic energy and motivation and works on the pleasure principle, seeking immediate gratification of instinctive need. The EGO is the executive branch of personality and operates on the reality principle, attempting to control the ID by postponing action until desired object can be achieved. The SUPEREGO is the most recent part of personality and developed by socialization in childhood. It is the moral/judicial branch of personality, enforcing control over the EGO by rewards and punishments such as anxiety and depression

Both Eysenck and Cattell’s models are based on statistical analysis. Eysenck proposed three dimensions of personality – Extroversion-Introversion/Neuroticism-Stability/Tough – Tender Mindedness. Cattell utilized a technique known as Factor Analysis to identify 16 traits that he considered best described personality.

The Social learning approach centres on the role of observation and imitation in the development of personality. It proposes that people mimic behaviours observed in others, referred to as ‘models’. It suggests, for example, that children will act more aggressively if they see others acting this way and that boys and girls may display different behaviour because they have different role models.

The Humanistic view is concerned with self-development, higher human motives, the acquisition of knowledge, understanding and aesthetics. This approach is particularly popular amongst psychotherapists.

Questionnaires associated with these approaches

Though not a psychoanalytic tool, the MBTI (Myers Briggs Type Indicator) is very much based on the work of Carl JungTAT (Thematic Apperception Test)

EPI (Eysenck Personality Inventory)16 PF (Cattell)OPQ (SHL)

FIRO-B

13

2.1.7 Approaches to Personality: Type Vs. Trait Theories

Many theories of personality can be divided very simply into those identifying distinct types and those which see personality as a combination of traits.

TYPE TRAIT

Type theories divide people into distinct groups, generally consisting of a combination of behaviours. The MBTI – Myers-Briggs Type Indicator, based on Jung’s typology, is a type theory. As are Belbin’s team types.

Essentially, all type theories classify an individual as one of several types of person, showing particular combinations of characteristics.

Where type theories stress similarities, trait theories stress differences between individuals. Trait theories potentially give more flexibility of description, as far fewer people will have a similar combination of traits than fall into a particular type.

The EPI, 16PF and OPQ are all based on trait theories

It is possible to go from one to the other; a combination of traits can lead to classification under a certain type, while how strongly inclined someone is towards a particular trait can lead to splitting up those with stronger and weaker preferences into two different types. For example, a combination of OPQ scales (Traits) can be used to predict a team type from the Belbin Model

Exercise 3 – Indicate what you see as the key difference between the psychoanalytic, social-learning and humanistic approaches in describing and explaining differences in personality.

Exercise 4 – What do you think are the relative advantages and limitations of the type and trait-based approaches to personality assessment.

TYPE TRAITAdvantages Advantages

Disadvantages Disadvantages

14

This is the end of the Section 2: Module 1.

Look back over your answers and consider how happy you are with them. If you are satisfied with them you are ready to move on.

15

SECTION 2: MODULE 2 - LEARNING OBJECTIVES:

To consider the assumptions about personality underlying the OPQ.

To gain an appreciation of the sorts of factors that influence personality.

To consider which aspects of personality are more variable over time.

16

2.2 Assumptions about the Nature of Personality

The definition and structure of personality used in the OPQ is based on some general assumptions about the nature of personality which can be elaborated as follows:

2.2.1 Individual Differences

People vary in their behaviour. That is to say, there are demonstrable individual differences between people.

2.2.2 The Stability of Personality

Personality is a changing thing but still shows certain stabilities. This recognises the day-to-day variations we see in individual behaviour. Despite these variations there will be certain behaviour styles which some individuals are likely to find difficult to adopt. In this regard, the desired personality for given job activities can be difficult for some individuals to maintain.

2.2.3 The Nature-Nurture Debate

There is a continuing debate among psychologists over the relative importance of genetic and environmental (e.g. cultural, educational) influences on personality. Of course, it is very difficult to separate the two, as those who inherit similar genes from their parents tend to experience similar environmental influences. One traditional method of attempting to split up those influences has been to study identical twins, separated near to birth and brought up in different homes. Identical (monozygotic) twins share the same genes but would, in theory, be exposed to different environments in these circumstances. Unfortunately, not only do very few such twins exist (a modern study by Thomas Bouchard in the USA could only find about 50 pairs), but their environments are unlikely to differ too much even under these circumstances.

Most psychologists believe that personality is open to both genetic and environmental influences. Some aspects, chiefly those that change little throughout life, are often considered largely genetic although, of course, genetic influences do not necessarily have to manifest themselves equally throughout life - some genetic diseases do not show symptoms until middle-age, for example. Extroversion is considered by some psychologists to be under strong genetic influence, though it is easy to see how some aspects of it could be affected by environment (were you encouraged to express yourself at school etc.?) Aspects of personality relating to beliefs and attitudes (e.g. how traditional you are) are perhaps more affected by environment and probably more prone to change in the longer term, which can affect their stability (test-retest reliability) over several years.

With regard to the nature-nurture question, the OPQ approach is interactionist, i.e. behaviour is an interaction between constitutional influences. All behaviour is potentially influenced by environmental influences but constitutional differences can obviously be responsible for those of similar background behaving in different ways. Influences which are more constitutional than environmental are the genetic, physiological and hormonal.

17

2.2.4 State versus Trait Characteristics

Some aspects of personality are inevitably more stable, less likely to fluctuate over time than others, which will depend more on an individual’s mood at the time. Optimism, for example, is a characteristic which seems to remain very stable over time (optimists remain optimistic, pessimists remain pessimistic). However, one’s mood is likely to be far more dependent on circumstance. If you have just had bad news, you may be feeling sad, despite being naturally optimistic.

Highly stable aspects of personality are termed traits, while those which largely depend on mood are called states. One area in which the distinction is very important is in the study of anxiety.

Those of high trait anxiety are generally tense as they go through life, continually feeling under pressure and having difficulty ‘switching off’. Those of high state anxiety will only get worried or keyed-up in particular situations, perhaps if they have an important meeting, presentation or test to face that day or if something has gone wrong at work. An individual may be low on trait anxiety (generally relaxed) but high on state anxiety (nervous or agitated in certain situations).

The OPQ32 Relaxed scale is designed to measure trait anxiety, while the Worrying assesses state anxiety.

A summary diagram of the potential influences on a human adult personality is set out below:

Influences on Human Adult Personality

18

Physiological Genetic

Constitutional

Educational

Family

Age

Life Experience

Situational

Socio-cultural

Exercise 5 – In your opinion, or from your experience, which aspects of personality may be largely genetic and which are affected by environmental influences?

Exercise 6 – How would we isolate which aspects of personality are genetic and which are environmental?

Exercise 7 – Which TWO of the following statements are true?

A. Personality is entirely inherited.B. Genes play no role in personality development.C. Personality is determined by a combination of many factors.D. Your first five years determine your personality for life.E. Personality can be modified by experience throughout life.

This is the end of the Section 2: Module 2. Think about your answers to the activities above and if you are content with them, then move to Module 3.

19

SECTION 2: MODULE 3 - LEARNING OBJECTIVES:

To gain an overview of the various methods that have been used to assess personality.

To consider the advantages and disadvantages of each of these methods.

20

2.3 Methods of Assessing Personality

A wide variety of techniques exists for assessing personality characteristics. They range from more objective methods to the more esoteric and unconventional.

2.3.1 The Interview

Probably the most common method of assessing personality is that of personal contact and discourse with the individual, often formalised in an interview. The problems of conducting systematic, thorough and unbiased interviews are well documented.

Although the use of a clear specification and structure in the interview can improve the objectivity of the process, influences like contrast, recency and halo effects, and the tendency for interviewers to make initial, impressionistic judgements all contribute to make interviews less objective and reliable than we would wish. In addition, valid judgements of personality are not always easy to achieve in an interview situation and decisions are likely to be influenced by the personal experiences, preferences and prejudices of the interviewer.

2.3.2 Physical Characteristics

Attempts to establish the personality correlates of different types of physique began with the work of Sheldon (1940, 1942) who proposed three main body types and corresponding temperaments. Although very few people are actually pure types, the muscular, athletic Mesomorph was described as essentially aggressive and risk taking; the short, plump Endomorph was essentially extrovert and fun-loving; and the thin Ectomorph as essentially shy, introverted and inhibited (see Figure 1). Independent research has shown some support for Sheldon’s work but correlations have been low and not usually of practical significance.

Figure 1 Physical Characteristics Associated with Personality

ENDOMORPH MESOMORPH ECTOMORPH

21

ExtrovertFun-lovingOutgoing

AggressiveRisk-taking

Athletic

IntrovertInhibited

Shy

Modern scientific techniques could allow us to go one step further and take physiological measures to assess personality, such as levels of neurotransmitters, electrical impulses through the skin, or electrical activity within the brain itself. This, however, would undoubtedly prove threatening to candidates, as well as impractical in a work setting. While such techniques may be helpful in some research and clinical diagnostic situations, they are not appropriate for the occupational user and still possess a good deal of subjectivity in interpretation.

2.3.3 Projective Techniques

Another method of assessing personality is through the use of various projective techniques in which, for example, candidates are required to interpret a formless shape or a semi-ambiguous picture, or perform a word association task, the idea being that they will project their personality in their responses. Responses are then translated by skilled interpreters into underlying personality traits. Probably the best known of these are the Rorschach inkblot technique and the Thematic Apperception Test (TAT). Such instruments are said to provide a rich source of evidence on emotions, attitudes, and cognitive processes. There is little evidence that these methods predict managerial success at all well. They are also unlikely to appear relevant and acceptable to candidates and can suffer from a good deal of subjectivity of interpretation.

2.3.4 Application Forms, References and Biodata

Personality characteristics are often inferred from information on the application form, particularly from areas like the interests section, e.g. those engaging in sports may be thought more competitive and active, or social activities might be taken to suggest a more outgoing manner. Needless to say, these assumptions need to be substantiated by other techniques. Not every competitor plays team sports for the enjoyment of being in a team.

References, for their part, can be subjective and many organisations will provide little more than the individual’s job title and how long he/she was employed. Many recruitment specialists indicate that references are often more informative if conducted by telephone rather than letter.

Biodata involves the use of objectively scorable application forms where questions are presented in some detail, often in multiple-choice format. Biodata forms need to be job-specific and to avoid the trap of self-fulfilling prophecies. The biodata approach has shown moderately good correlations with job success and can be especially useful when dealing with large numbers of applicants.

2.3.5 Astrology, Phrenology and Graphology

Although personality is concerned so directly with how we behave, a number of unusual methods have been suggested to measure it. Many astrological forecasts involve a description of the personality types typical of different birth signs. In the past, even the bumps of the head (phrenology) have been seriously suggested as a method of assessing personality. A very small percentage of employers use graphology (handwriting analysis) as a technique for assessing personality. There is little scientific evidence to show that these techniques are successful.

2.3.6 Simulations

Simulations such as group activities and presentation exercises are used quite extensively, particularly at management level, to assess such characteristics as “leadership”, “interpersonal skills” and “sensitivity”.

22

Group selection methods are often popular for jobs that require a good deal of face-to-face contact with other people. Assessing personality characteristics from group exercises has the problems of how behaviour is evaluated, the dynamics of the particular group people find themselves in, and the appropriateness of the task they are asked to tackle. Nevertheless, simulation exercises are useful indicators of success in selection and job promotion. Analysis suggests that there are three primary factors which can be assessed in group exercises. These are prominence, practical problem solving and sensitivity to others.

2.3.7 The Questionnaire

Probably the most frequently used method of assessing personality, other than subjectively in interviews, is to use some kind of personality questionnaire or inventory.

The credit for the first personality questionnaire to be devised as a placement or selection tool is usually given to Woodworth and his Personal Data Sheet which was designed to be used as a screening device for neurotic behaviour on soldiers in the US Army in World War One.

Over the next thirty years there was an abundance of questionnaires and inventories designed to measure or assess personality by self-report. Inventories were seen as a replacement for observation and concentrated mainly on self-reports of behaviour. Subsequently they became much broader and were designed to describe separate aspects of behaviour or ‘traits’ such as friendliness, confidence, persistence, etc. Dozens of instruments were constructed, but perhaps the best known of these early ones is the Bernreuter Personality Inventory which gave scores on Neurotic Tendency, Self-Sufficiency, Introversion and Dominance.

There then followed a period when statistical approaches predominated and personality theory became secondary to the search for personality dimensions. Guilford and Thurstone were at the forefront of this approach, which again generated a vast number of questionnaires. Much debate took place about how many personality factors can be reliably measured, how these should be organised, and the most appropriate names for them. The methods were used in Eysenck’s three factor model (Extroversion, Neuroticism, and Psychoticism) and in Cattell’s 16 factor hierarchical model.

Perhaps the most widely used of any personality questionnaire was the Minnesota Multiphasic Personality Inventory (MMPI) (Hathaway and McKinley, 1945). Intended as an aid to psychiatric diagnosis, this was constructed on an empirical or “criterion keyed” basis using patients from different diagnostic groups to determine item selection for its various scales. It is thus not particularly suitable for use on wider populations. A descendant of the MMPI, the California Personality Inventory (Gough, 1957), is also criterion keyed but emphasises criteria of social and intellectual effectiveness. It has been widely used in the United States, but far less in the UK.

23

Such an abundance of personality questionnaires have been devised that Cattell in 1973 commented that they have been the ‘prolific rabbits of the psychometric world’. A comment by Sir Frederick Bartlett illustrated the scepticism surrounding the use of personality questionnaires, and their reputation with many psychologists, when he said “I don’t know... they may be all right... they always seem to me to over-estimate the self-knowledge of the subject and to underestimate his sense of humour.”

Many questionnaires have been devised for research purposes to test out a researcher’s particular hypothesis or theory and many others such as the MMPI are really suitable only for clinical use.

Relatively few questionnaires have been devised specifically for occupational use, and those that have are on the whole not very well researched or validated. However, some reputable questionnaires such as the Guilford Zimmerman Temperament Analysis, the Gordon Personal Profile and Inventory, and Cattell’s 16PF have been successfully used in the occupational area, and have been found to be useful in selection, development and counselling situations.

The 16PF was developed in the USA during the 1940s and 50s largely on school and college students and provided a wealth of new ideas about the structure and functioning of personality in normal subjects. It was based on factor analytic methods and proposed sixteen factors which were claimed to be functionally rather than statistically independent.

2.3.8 So how valid a measure are questionnaires?

Figure 2 A Practitioner’s Guide to Assessment Techniques

Although line managers do not usually list ability characteristics as the prime factor in job success, ability tests, particularly of the work sample type, follow on the list of valid methods. The evidence for the validity of ability tests is consistent across a wide range of studies.

Next in terms of validity are personality questionnaires, group activities, biodata, competency based interviews and in-trays. The most popular of these methods is the personality questionnaire, probably because of its ease of use and scoring.

24

Whilst there is some remaining controversy about its validity, research evidence to support its use is increasing.

There is certainly the dilemma that meta-analysis (integrating and analysing data from a combination of studies) of validity coefficients does not show questionnaires of personality to have the high empirical validities suggested by their popularity. However, much of the early research has been conducted on old and clinical instruments rather than occupational questionnaires.

Of rather lower validity comes unstructured interviewing, with graphology and astrology giving no better than chance prediction.

Overall, questionnaires generally compare favourably with other methods of assessing personality in terms of validity. As will be discussed during the OPQ Training, research findings indicate that the OPQ stands up particularly well in this respect.

Exercise 8 – What is Personality and what influences it?

Your Task

Section A:

Work through questions 1-5. Do not spend too long thinking about any one area.

These questions ask you to focus on your own personality, its enduring features and aspects that have been more susceptible to change. You are then asked to look at how your personality impacts on your career choices and behaviour at work.

1 What key words would you use to describe your personality?

2 What and who has influenced the formation and development of your personality?

3 What aspects (if any) of your personality have changed?

4 To what extent and in what ways has your personality influenced your own decisions and choices (i.e. career choices, other choices)?

25

5 Which aspects of your personality are most and least relevant at work?

26

Exercise 8 – Section B

Your Task

Complete the gaps in the following information on personality theory.

The following person specification covers all these elements (personality, values, beliefs, motivation, attitudes, interests and abilities) although you may find some of the distinctions are blurred.

Under the heading ‘element’ please classify the requirements in the person specification.

Job Title: Youth WorkerEmployer: ABC Community Centre

Requirement ElementLikes to initiate contacts and spend time in the company of others. Preferably a natural extrovert PersonalityHas a deep respect for cultural differences

Enjoys participating in and following sports/physical activities

Can manage an annual budget effectively

Can design a variety of activities to stimulate young people Ability

Accepts and is committed to live by the ABC doctrines

Takes the Health and Safety aspects of the role very seriously

Prefers to plan ahead

Finds seeing young people grow and develop rewarding

This is the end of Section 2: Module 3. In Module 4 we will go on to look at the different versions of the OPQ in more depth.

27

SECTION 2: MODULE 4 - LEARNING OBJECTIVES:

To become familiar with the different members of the OPQ family.

To start to look at the potential uses of these.

To begin to become familiar with the aspects of personality the questionnaire measures.

28

2.4 Overview of the OPQ Family (optional after 2.4.3)

2.4.1 The Occupational Personality Questionnaires (OPQ)

The OPQ family is designed to assess the typical or preferred behaviour of individuals in a way that is relevant to the world of work. Researched and developed in the early 1980s and regularly updated, OPQ was first introduced in 1984 in Britain and is now used worldwide, having been adapted and translated into many languages.

2.4.2 OPQ Structure

OPQ is based on the definition of personality as that which is concerned with a person’s typical or preferred way of behaving, thinking and feeling.

We propose that personality is concerned with three main areas or domains:

The Relating Domain – how an individual relates to others, characterised by such traits as assertiveness, outgoingness and empathy.

The Thinking Domain – how an individual typically thinks, covering traits like conservatism, abstract thinking and detail consciousness

The Feeling Domain – the emotions, including traits like anxiety, tough-mindedness and optimism.

Potentially there is a fourth area - the Energies Domain - characterised by traits like vigour, competitiveness and decisiveness. However, Energies impinge upon the three other domains. For example, one of the implications of competitive drive might be the wish to put others down and decisiveness is clearly related to an individual’s thinking style. Moreover, traits like the need for affiliation (Relating Domain) and the need for variety (Thinking Domain) can in themselves be seen as energies.

The OPQ Model of Personality is illustrated in Figure 1 below:

Figure 1

29

FeelingDomain

RelatingDomain

ThinkingDomain

Energies

Versions of OPQ (optional)

The first version of OPQ to be developed was the Concept Model, which divides personality into 30 deductively-derived scales representing aspects of typical behaviour concerned with relationships with people, thinking style and feelings and emotions.

2.4.3 OPQ32

This has been developed to take account of changes in the world of work over the past few years including the need for an international model of personality. This version, called OPQ32, comprises 32 scales. It offers both a normative scoring approach (OPQ32n) which employs a rating scale, and an ipsative scoring approach (OPQ32i) which forces people to make a choice between most and least preferred aspects of their personalities. Both approaches have points to recommend them, but the ipsative is thought to be more effective in controlling distortion due to response bias, while the former imposes few ‘constraints’ on the respondent. (Response bias will be explored on the course).

Other versions of OPQ were developed by factor analysing the larger versions and producing ‘summaries’ of this model to varying extents

2.4.4 Factor Solution

A 16 factor model questionnaire (Factor Solution) has been developed. This offers a normative (FS5.2) and an ipsative (FS4.2) version.

2.4.5 Customer Contact Styles Questionnaire (CCSQ)

These versions are based on more specific areas of work and have been designed following job analysis of these areas. The Customer Contact Styles Questionnaire (CCSQ) forms part of the ‘Customer Contact’ portfolio of assessment techniques and includes normative (CCSQ5.2) and combined normative/ipsative (CCSQ7.2) versions. The Customer Contact Styles Questionnaire is designed to measure the 16 dimensions of personality which have been found to be important in non-supervisory sales and customer service roles

The Customer Contact portfolio also includes an Aptitude Series, 360º competency rating instrument, interview guide and application form to provide a streamlined range of assessment tools for sales and customer service jobs.

2.4.6 IMAGES

IMAGES is the most broad, and also the most swiftly administered and scored version of OPQ, and is SHL’s nearest equivalent to a “Big Five” model of personality. A 360 version of this, known as IMAGES II is available, allowing self perceptions to be compared with those of colleagues. The number of factors affect how much detail the questionnaire gives and how long it takes to complete.

30

Exercise 9

Your Task

Look back at the words you wrote for Exercise 8 Section A describing your own personality. Then look in the Scale Descriptions booklet in the OPQ32 section.

Which of the OPQ32 scales does each of the words you wrote best relate to?

Now look through the rest of the booklet describing the other questionnaires in the OPQ family and locate the relevant scale.

How important is this aspect of your personality for performing well in your job?

31

This is the end of Section 2: Module 4. We will be looking at the various versions of the OPQ and interpretation of the scales in much more depth during the course.

32

SECTION 2: MODULE 5 (Optional) - LEARNING OBJECTIVES:

To gain an appreciation of how personality questionnaires are developed.

To look at how the OPQ in particular was developed.

33

2.5 The Development of the OPQ

Since the launch of the OPQ in 1984, continual investment has been made in further research and development. This has included norm updates, validation, design of computer-based options and design of new OPQ versions.

Full details of the development process for each OPQ version can be found in the relevant OPQ User Manual.

2.5.1 Why was the OPQ developed?

There were 4 main reasons behind the creation of the OPQ

1. To design a questionnaire that was relevant and acceptable for use in the workplace.

2. To produce a questionnaire that avoided unnecessary jargon and focussed on what people typically do.

3. To provide a comprehensive range of personality scales, even if this did entail some overlap of measurement.

4. To provide a questionnaire that could be used by appropriately trained HR practitioners.

2.5.2 Approaches to the development of Personality Questionnaires

Deductive and Inductive Approaches

A deductive or conceptual approach is one in which questionnaire items are written to reflect an a priori model of personality. The OPQ Concept Model of Personality was developed to provide this detailed description and is based on a rational analysis of the important personality variables listed by organisations in interviewing, appraisal and assessment centre documentation.

An inductive approach is one where a wealth of data about personality is generated, and a reductionist approach is then taken to establish the simplest, most straightforward statistical solution to the data, using techniques such as Factor Analysis.

Factor Analysis is a technique based on the statistical manipulation of correlations. In terms of personality questionnaire, the key aim of factor analysis is to identify a set of scales independent scales or factors. Not surprisingly stringent factor analysis is likely to produce fewer scales.

Factor analysis on the Concept Model scales has been used in the development of other versions of the OPQ consisting of a reduced number of scales that are more independent of each other. Current versions of the OPQ developed through factor analysis of the Concept Model, include the 16-scale Factor Solution and the 6-scale IMAGES. Just as the Concept scales had to satisfy considerations of statistical adequacy, the scales of these instruments were also guided by issues of psychological meaningfulness. These models represent a robust statistical summary of the Concept Model. Fewer scales mean less detail, but fewer questions and thus shorter administration time.

34

2.5.3 Stages in the initial development of the OPQ

STAGE 1 - INITIAL RESEARCH

Review of all existing questionnaires and models of personality relevant to work.

Review of all validation data on the relationship between personality scales and job performance.

Review of documentation from sponsors and other client organisations to investigate which aspects of personality were relevant to them.

Repertory grid study.

STAGE 2 - CONSTRUCT ADJECTIVE CHECKLIST

For each scale, 6 definitive adjectives were chosen - 3 on each of the positive and negative poles of each scale e.g.

Sociable -------------- Shy (Socially Confident).

STAGE 3 - FULL LENGTH ITEM TRIALLING

Over 2000 items were generated, of which 1500 were actually trialled. A good item was taken as one which was closely related (i.e. had a high correlation) to other items in its own scale but was not closely related to items in other scales.

STAGE 4 - STANDARDISATION STAGE

Item Analysis – Conducting a final check on the reliability of the model and items measuring each scale.

Reliability – To obtain internal consistency and alternate form reliability for every scale.

Norms – To produce sten norm tables, and investigate the relationship of scores with demographic variables.

Design – Production of workable administration instructions, materials and scoring systems.

35

2.5.4 Stages in the development of OPQ32

STAGE 1 – ESTABLISH DEVELOPMENT OBJECTIVES The objectives were to update the original concept model and build a new

international model of personality. Achieve even better levels of validity. Improve the reliabilities of some scales. Reduce the overlap between scales. Keep the questionnaire length to a minimum.

STAGE 2 – DATA GATHERING AND REVIEW Quantitative data was collected on the existing concept models in the identification

of data sets that could provide additional useful information. Criterion related validity – Since the initial launch of the OPQ, SHL has collected a

wealth of validation data showing consistently significant relationships between OPQ scales and different management competencies.

Construct Validation – The relationship between the OPQ Concept and other existing questionnaires including: 16PF5, NEO-PI-R, SHL’s MQ Motivation Questionnaire and SHL’s CCSQ Customer Contact Styles Questionnaire.

Qualitative – An international consultation exercise was carried out with experienced OPQ users from across the world to evaluate the cultural issues that could affect responses to items and therefore scales.

STAGE 3 – PROPOSE NEW MODEL Using the information already gathered, a panel of experts proposed a new model

of personality. High and low scale descriptions were given, and reviewed scales presented.

New items were written for the new scale descriptions. These items had to be grammatically simple, work relevant, use accessible vocabulary and non-ambiguous.

STAGE 4 – TRIAL NORMATIVE QUESTIONNAIRE From an initial pool of over 500 items that were trialled, 230 were chosen for the

final questionnaire.

STAGE 5 – REVISE MODEL AND FINALISE NORMATIVE QUESTIONNAIRE (OPQ32n)

16 scales contained 6 items per scale and 17 (The 33rd scale is Social Desirability) scales contained 8 items. Items were selected on the basis of their strong internal consistency and good variance.

STAGE 6 – DEVELOP IPSATIVE QUESTIONNAIRE (OPQ32i) After the model was finalized and items selected for the normative questionnaire,

OPQ32n, the ipsative model was developed. Although the scales remained the same, slightly shorter items were used. These

were trialled and then presented in the quod format whereby respondents choose 1 item that is most typical of them and 1 item that is least typical. There are 416 items (104) quods in OPQ32i.

36

Exercise 10 - The development of the OPQ and Reliability

1. To produce a questionnaire with occupational relevance, what were the four activities carried out during the preliminary development stage of the OPQ Concept Models?

2. Consider the following, delete the incorrect words and fill in the gaps:

Factor Analysis is a process in questionnaire construction. It is used to develop a reduced number of / better / more sophisticated scales that are dependent / independent / interdependent of each other. The output of a factor analysis of personality data is parcels of related items to produce a complex / conceptual / simple solution that allows meaningful interpretation.

3. Personality scales must be psychologically meaningful and measurable. What was the main feature required of the OPQ scales during the final development stage?

This is the end of Section 2: Module 5. In the next section we move on from looking at the instrument itself to thinking about how it may be used.

37

SECTION 2: MODULE 6 - LEARNING OBJECTIVES:

To begin to think about the different ways in which the OPQ may be used.

38

2.6 Applications of the OPQ

2.6.1 The Role of Competencies

Competencies have become an increasingly important component of the way many organisations manage their people. Competencies define the most important personal characteristics required to be effective in a role (such as Decision Making, Resilience, or Communication) and as such form the basis of many selection, development and performance management processes.

There is no agreed definition for what we mean by competencies, and this has led to a considerable degree of argument and confusion over the years. Richard Boyatzis, whose seminal book “The Competent Manager” popularised the Competency approach in the 1980S, defined competencies as follows:

“A job competency is an underlying characteristic of a person which results in effective and/or superior performance in a job .... it may be a motive, trait, skill, aspect of one’s self image or social role, or body of knowledge which he or she uses.”

However, a perhaps more comprehensible definition might be that “competencies are those attributes of an individual which are important for effective performance in their role, and which can be observed through their behaviour. These attributes include all the knowledge, skills, personality and motivation they have which are relevant to their work.”

Typically, between 8 and 12 competencies are required to describe the important attributes required to perform a role effectively. A typical competency model for a management role might look like this:

Commercial Acumen Resilience Planning & Organising Drive & Determination Decision Making Creativity & Innovation Analytical Thinking Teamworking Communication Interpersonal Sensitivity Change Orientation Specialist Knowledge & Skills

39

Each of these competencies would be further defined, giving specific examples of positive and negative behaviour associated with the competency. For example:

TeamworkingDefinition: Builds effective working relationships with others, co-operates with colleagues to achieve goals and is sensitive to their needs.

Positive behaviours Negative behaviours Encourages others to contribute Prefers to work alone Asks others for their ideas Promotes own ideas over others’ Builds on others’ contributions Criticises others’ contributions Gives priority to team goals Gives priority to meeting own

needs Celebrates team success Does not consider impact on

others

Since competencies form the basis for much HR practice in organisations, and personality is one of the factors determining behaviour on a given competency, the OPQ is often used for many applications linked to competencies. The use of the OPQ in this way is supported by validity evidence. Gibbons et al (1995) found evidence that ratings on relevant OPQ scales predicted performance on 16 management competencies. For example, performance on the competency of “Creativity and Innovation” was predicted by a combination of OPQ scales of Independent, Traditional (negative correlation), Conceptual and Innovative on the OPQ Concept Model.

2.6.2 Selection

The OPQ can be used in a variety of ways to help make effective selection decisions. Validity evidence suggests that the OPQ will be most effective if it is used in a structured manner, i.e. selecting relevant aspects (scales) to focus on in relation to the competencies/job requirements. When used in such ways, there is extensive evidence that the OPQ can be used to effectively predict people’s effectiveness in a number of occupational settings. The OPQ should be used alongside other information about the candidate, rather than in isolation.

To Aid a Selection Interview

The applicant’s profile can be used to structure the interview, with the emphasis on those attributes identified as being most relevant to the selected competencies. Relevant aspects of the profile can then be probed, e.g. if the job requires planning skills, the candidate’s profile areas relating to structure can be probed to gather behavioural examples. The mapping of the OPQ onto the competencies is often based on a combination of prior research, detailed knowledge of the job and the judgement of OPQ experts. Mapping is most effectively established by conducting a localised validity study. A comprehensive picture of the individual’s suitability can be built up by relating the profile to track record and other relevant information.

40

As an Input to an Assessment Centre

The OPQ is frequently used as one input to a multiple assessment procedure, or Assessment Centre, to support or clarify other sources of information, and is often supplemented by a feedback interview. Many organisations will select the OPQ scales relevant to each competency/criteria in advance of the Centre. This will then be used as a basis for summarising the candidate’s likely performance on these competencies according to their OPQ profile, as one source of evidence at the post-centre integration session. The information from the OPQ can help to confirm the results from other exercises or highlight where the observed behaviour on the Centre does not tally with the person’s self insight, e.g. the candidate is observed as being very creative in a group exercise but sees herself as much less ‘Innovative’ than others on OPQ. This can help assessors identify areas where they should look for more information to explain the difference.

Statistical Validation

Where empirical validation studies can be carried out, this lends extra weight to the interpretation of the OPQ profile for selection. Predictions about an individual’s likely suitability for a particular job can be made, by using a prediction (or specification) equation. Some organisations have drawn up “danger zone” profiles which highlight the positive and negative features of the profile in a structured way, to be followed up at interview. For more information on these techniques, see the chapter on Validation.

2.6.3 Development, Counselling and Team Building

The OPQ is a very powerful tool in developing people in the workplace. The range of potential development applications is large, and includes:

Individual development and behaviour change Counselling Team building and development Leadership skills and management development Specific skills development Management of change Coaching

Individual Development and Behaviour Change

Self-AwarenessFeedback of a personality profile often provides the individual with valuable self-insight and increased self-awareness. This is particularly beneficial if it can be linked to job performance, perceived strengths and weaknesses, or training and development needs.

Development CentresIn such applications the OPQ may be used alone as the basis for a one-to-one feedback and development planning discussion. Alternatively, it may be used to complement other information, e.g. in a Development Centre or as part of a development and training workshop.

41

Counselling

Self-AwarenessThe OPQ can be a valuable tool in working with individuals to help them to arrive at a more objective and comprehensive picture of themselves in order to take responsibility for taking action or making choices. The OPQ can help to clarify how their style relates to current organisational requirements; how effectively they are currently performing; and how they might develop their style.

Career Choice and Job SatisfactionThe OPQ also contributes to an understanding of how personality influences career or job choice (helping people to choose jobs which are more likely to suit their personality style); and likely job satisfaction and effectiveness in that field.

Stress Counselling and ManagementStress can be defined as occurring when the perceived demands on a individual tax or exceed that individual’s perceived ability to cope. Clearly, there are marked areas of commonality in what people regard as major sources of stress in the work environment. How different individuals react to these potential stressors is likely to vary quite substantially. Personality differences render individuals more or less vulnerable to general and specific sources of stress in particular, i.e. something that one person finds stressful may appear non-stressful to another person.

There is reason to believe that certain combinations of extreme OPQ scores may be indicative of a generalised vulnerability and increased susceptibility to the negative effects of stress. Conversely, a greater degree of resilience can be inferred from the opposite trends in scores. This relates to findings described by the “Hardy Personality” and “Type A and B” behaviour. Particularly relevant in this context are scores on OPQ dimensions relating to:

General levels of anxiety and tension Proneness to worry excessively about specific events Sensitivity to criticism and hostility Optimism vs. pessimism Self confidence/self esteem

Of course, it is not only those who are typically anxious, sensitive and depressed who are subject to stress. Stress is likely to be experienced by any individual, finding him or herself in a situation where there is a poor person-environment fit. On the other hand, a good person-environment fit, characterised by compatibility between environmental demands and the personal attributes necessary to meet those demands, increases resistance to stress, and the likelihood of job satisfaction and a sense of personal well being. In this context, each and every one of the OPQ dimensions becomes relevant to identifying potential sources of stress for a given individual in a given environment, and, perhaps more importantly, for identifying potential strategies for coping more effectively. This underlines the need for careful job analysis and clear job descriptions and person specifications. Time spent on these endeavours is likely to be time well spent: employing good selection procedures in which every effort is made to fit round pegs into round holes is arguably one of the most fundamental ways of reducing organisational stress.

N.B. It should be noted that the OPQ is designed as a occupational tool and as such, the limitations of the information must be respected, i.e. it is not designed to diagnose clinical issues. In using the OPQ in counselling applications, both this fact and the skills and expertise of the counsellor should be borne in mind in establishing boundaries for the discussion.

42

Team Building and Development

The OPQ provides information on an individual’s preferred style of behaving at work. Most people do not operate in isolation - thus their preferred style and the interaction of this with that of others impacts on the functioning of teams in which they operate. Self-awareness and a comparison of people’s OPQ profiles can thus be helpful in development of the team.

A number of useful outputs, based on specific combinations of scales, can be generated from the OPQ. As well as the well-established Belbin Team Types model which explores an individual’s preferred role in a team, a new output, the Team Impact model, can also be generated. This pays particular focus on an individual’s likely preferred role in a project team. These outputs can aid in the understanding of conflicts and help to develop the team, e.g. by encouraging members to adopt a balance of roles to achieve maximum effectiveness. For more information, see the chapter on Leadership and Teams.

Leadership Style

The OPQ can be used to infer a person’s likely style of management, and also how a person is likely to prefer to be managed. For example, someone who enjoys taking charge and influencing others, but is not interested in the views or feelings of others would probably take more of a directive approach to managing people; someone who is very independent-minded and goal focused may respond best to being managed in a more “hands-off” manner.

The use of the new Leadership Potential Indicator, which is derived from the OPQ, provides a very powerful tool for evaluating an individual’s potential across a range of leadership and management domains. Information about an individual’s leadership and reporting style can be used to build a person’s self-awareness and thereby enhance their effectiveness at work.

Specific Skills Development

Of course, the potential applications of the OPQ extend beyond leadership skills development. Other areas in which the OPQ can be used to increase self-awareness as a lever for behaviour change and skills development include: influencing skills; problem solving and creativity; and coping with change (see below).

43

Management of Change

In many organisations, change is almost becoming the norm. Helping managers to deal effectively with change is thus a key priority. The OPQ can help in several ways here. The OPQ can be used with individual managers to increase their self-awareness of how they are likely to react to change and also see ways in which they need to adapt their behaviour to move with organisational developments. It can also be used by those managing the change process to help identify areas in which particular managers are likely to block or to be effective agents for change, so that this understanding can be used to facilitate change.

2.6.4 Restructure, Redundancy and Outplacement

Another organisational reality in today’s business climate is that of restructure and redundancy. There are some clear parameters surrounding the use of OPQ (and other psychometrics) in such situations. In situations of straight redundancy the OPQ should not be used for decision making - such decisions are best made on the basis of on-the-job performance data.

Where organisations are restructuring and new positions created, the OPQ can be effectively used to help make positive selection for appointment decisions, much as with normal selection (see above).

In terms of outplacement, as with career counselling (see above), the OPQ is valuable in aiding self awareness, helping managers to identify what types of careers and roles are likely to provide more satisfaction and suit their preferred style. There is also research evidence suggesting that the OPQ can be used to identify areas in which people may need support in coping with redundancy and applying themselves to job search activity.

2.6.5 Research

The OPQ has formed the basis for a number of research projects investigating personality at work, and differences in personality between occupational groups. In March 1990 a major research project was undertaken by SHL to investigate personality characteristics within the general population. This study, the largest of its kind ever undertaken in this country, provides valuable information about sex differences, age trends and differences between different racial groups in personality terms. As a rigorously researched tool, the OPQ represents an objective, systematic and reliable method for researching personality attributes, enhancing our understanding of the impact of personality in the workplace.

Think about how you are intending to use the OPQ in the future. You may wish to write down a few specific objectives you wish to have covered during the course. Then move on to the next section.

44

SECTION 2: MODULE 7 (optional) LEARNING OBJECTIVES:

To revisit the importance of standardisation in test administration.

To revise the process of OPQ administration.

To consolidate your knowledge of the key differences between test and questionnaire administration.

You may already be familiar with the administration of the OPQ in which case move on to the Exercise 11. If you are happy with your level of understanding, move onto the next section straight away.

45

2.7 Administration, scoring and norming the OPQ

2.7.1 Summary of OPQ Administration and Scoring Options

Figure 1

Administration

Scoring Norming &

Profiling

Interpretation

Hand-scoring

Profile charts

By trained user

Paper booklets &

answer sheets

SHL Bureau service

Desktop computer

Automatic

Automatic

Expert reports (PC)

Handheld PC Automatic

Automatic

Internet (client-

controlled)

Automatic

Automatic

Expert reports (Internet)

Internet (SHL Bureau service)

Automatic

Automatic

46

2.7.2 OPQ Administration

Whichever method of administration is being adopted, it is important to adhere to the correct administration procedures.

The OPQ on-line may be completed without local supervision subject to the following provisos:

It should not be released in an uncontrolled way to any candidate. For high impact situations, where there is no local supervision, appropriate

provision should be made for candidate authentication and validation of the results.

In all circumstances a qualified certified assessor should control who has access to results, under what circumstances and in what format.

The OPQ is not a timed exercise and it does not require strict adherence to timing or some of the other features of aptitude test administration. Nevertheless, strictly controlled administration is still essential for two main reasons:

For security purposes - to retain control of the materials.

For briefing purposes - respondents need to know exactly why they are being asked to complete the questionnaire, how the results will be used, and how to go about answering the questions. They need to be able to ask questions and to be given clear and accurate answers by a trained administrator. Potentially, the questions asked will be more difficult to answer and will require even more sensitivity than those which arise when aptitude tests are administered; therefore, the skill of a trained administrator will be essential.

2.7.2 Introducing the OPQ

The quality of the introduction to an OPQ session is likely to be critical to the quality of data collected. Poor introduction and administration is likely to increase the chances of respondents manipulating the responses they give.

If administering the OPQ in person, the style should be professional, clear and empathic, without becoming overly familiar or informal. The exact format of the introduction will depend on the nature of the session, but the following points should be included in some form:

This questionnaire is used in order to provide objective information on personality, which is relevant to successful performance in aspects of the job. The results represent a useful additional source of information, which aids the decision making process.

In the questionnaire you are asked to rate yourself on a number of statements. The questions are concerned with how you typically behave at work, so if you are

in any doubt try to answer from a work point of view. People who try to guess what they think is wanted are often incorrect, so just try

to rate yourself as accurately and honestly as possible. As its title suggests, this is a questionnaire, not a test. Consequently, there are

no right or wrong answers. It is concerned with personality style and not abilities.

You will have the opportunity to receive feedback on the results of the questionnaire.

There is no time limit, but you should work quickly rather than pondering at length over any one question. Please complete all the items.

47

In some situations, it might be appropriate to point out that a distortion or consistency check is included in the questionnaires.

2.7.3 Questionnaire Administration

Prior to the session the administrator should be familiar with all the materials and instructions, including computer administration options and Administration Card where relevant. The instructions for completing the relevant questionnaire should be read out aloud to the respondents. There should be the opportunity for respondents to ask questions about the completion of the questionnaire before they begin.

There are no time limits for completing the OPQ, but respondents should be paced to encourage them to finish in a reasonable time. Even when this is done, there is likely to be considerable variation in response times, and contingency for this should be built into the assessment timetable. At the end of the administration session all materials should be collected and reusable items will need checking before being used again. Respondents may have questions about their completion of the questionnaire before moving on to their next activity.

2.7.4 Scoring and Profiling

As indicated above, there are a range of options for scoring and profiling the OPQ. All questionnaires except OPQ32i, CM4.2, FS4.2 and CCSQ7.2 can be hand scored, and computer scoring and profiling options exist for most questionnaires. Raw scores are converted to Stens and are then profiled, usually on to a pre-normed profile chart. OPQ scores are normally represented on a profile in the form of sten scores. The sten scale runs from 1 – 10 where the mean is 5.5 and the standard deviation is 2. The sten scale is normally distributed, thus sten scores of 5 or 6 are typical of most people, whereas sten scores of 1 or 10 are representative of much fewer people.

2.7.7 OPQ Norms

Norms are sets of data derived from groups of individuals who have already completed a test or questionnaire. These norm groups enable us to establish where an individual’s score lies on a standard scale, by comparing the magnitude of response against that of other people.There are three main norm groups for OPQ32 – Managerial and Professional, Graduate and General Population norms.People vary markedly in their performances and qualities and, therefore, the norm group against which an individual is compared is of crucial importance. It is very likely for instance that the conclusions reached will vary when an individual is compared in two different groups; for instance, school leavers and managers in industry. For this reason, it is important to ensure that the norm groups used are relevant to a given group or situation.

48

Exercise 11 – Administration of the OPQ

Your Task:

Think back to ability test administration. Although both require administration by a trained person under standardised conditions, there are key differences between the administration of ability tests and that of personality questionnaires.

You are given a number of facts about the administration of ability tests on the table overleaf. Complete the table highlighting how the administration of personality questionnaires differs. Please refer to Section Three in the Pre-course work booklet where necessary.

KEY DIFFERENCES BETWEENABILITY TESTS AND PERSONALITY QUESTIONNAIRE ADMINISTRATION

ABILITY TESTS PERSONALITY QUESTIONNAIRESWhat they measure - job relevant skills (verbal, numerical, spatial, diagrammatic etc) and speed and accuracy of completion.

What they measure

Motivating and controlling for distortion - work as quickly and accurately as you can, discourage from cheating, advise candidates to avoid wild guessing, relax candidates with less formal introduction. Answers are either correct or incorrect.

Motivating and controlling for distortion

Sharing and Discussing Results (feedback) - legal requirement if storing data on computer. Recommended by SHL as courteous, good PR, ethical. Often difficult due to time and resourcing constraints.

Sharing and Discussing Results (feedback)

Time Limits – strictly timed with stopwatch and independent time. Different tests have different time limits.

Time Limits

Completing All Items - candidates may not complete all items in the time allocated. Told in admin to work as quickly and as accurately as they can throughout the test.

Completing All Items

49

ABILITY TESTS PERSONALITY QUESTIONNAIRES

Imposing Test Conditions – Imperative that test conditions apply for the duration. Candidates may not leave and re-enter the room nor may they talk or ask questions once the test has started.

Imposing “Test” Conditions

Hand and Machine Scoring - all hand or machine scored.

Hand and Machine Scoring

Profiling of Results - results can be profiled to give visual representation of how well a candidate has performed. However results do not need to be profiled.

Profiling of Results

Others

This is the end of the Section 2: Module 7. By now you may have completed your on-line questionnaire which will help to give you an idea of the approach to administration.

50

SECTION 2: MODULE 8 - LEARNING OBJECTIVES:

To look at how accurate measurement of personality is.

To consider the factors that affect the accuracy of measurement.

To look at ways of quantifying the level of accuracy.

51

2.8 Reliability

No instrument can measure with one hundred per cent accuracy. Error is a natural consequence of measurement. We know that instruments vary in their precision and therefore the degree of error which they produce. It is essential to know the typical band of error that an instrument gives in order to quote a tolerance expressed as a plus and minus figure around an estimated value.

Measurement of accuracy is of immense importance in psychometric testing. Personality questionnaires give scores, but they are not perfectly precise; scores should be regarded as estimates of an individual’s personality characteristics.

The definition of reliability as it relates to personality questionnaires is as follows:

Reliability is concerned with precision of measurement. Put another way, reliability involves the errors of measurement within the questionnaire and its administration and scoring.

Several factors influence the reliability of personality questionnaire data. These are similar but not identical to those which are important in relation to test scores. They are:

Figure 1:

52

CONDITIONS ERRORS OF MEASUREMENT

ADMINISTRATION

CONTENT TEMPORARY STATES

SCORING

Conditions: Poor lighting; inadequate space; distractions or noisy conditions disturbing concentration; lack of “pacing”, allowing some respondents to ponder too long on each question.

Administration: Lack of standardised instructions; examiner’s ‘throw away’ remarks; respondents not understanding what is required of them; poor briefing.

Scoring: Use of wrong key; not following the scoring instructions; addition errors; key badly aligned or designed; hand scoring systems for personality questionnaires are generally more complex than those for aptitude tests, and thus more likely to be subject to error.

Temporary States: Illness; fatigue; motivation; test sophistication; mood; trying to please.

Content: Ambiguous or vague items; poor design; item type.

Relationship between Reliability and Validity

Reliability is important because of its relationship with validity. The general rule is that an assessment method can be reliable without necessarily being valid; however, it cannot be valid and unreliable. That is to say that reliability sets the upper bound to validity and is a prerequisite of a valid measure.

The two principal issues in reliability are, first, to gain an estimate of the reliability of an instrument and, second, to use that information to set a band of error around individual scores.

2.8.1 Methods of Estimating Reliability

The reliability of a questionnaire scale is normally expressed as the correlation between two or more sets of scores on the same scale for the same group of individuals. Usually the product moment correlation coefficient is used. There are three main types of reliability:

53

2.8.2 Internal Consistency Reliability

A measure of the accuracy or consistency with which a set of questionnaire items measure one particular scale of personality.

The most commonly used method is Cronbach’s coefficient alpha, which can be represented as the mean coefficient of all the possible pairings of the items in the scale.

If the coefficient is too low then it suggests that the scale has very mixed or even ambiguous items.

Too high a coefficient implies a very narrow factor with items that repeat essentially the same idea.

Optimum alpha coefficients for personality scales lying in the range 0.6 to 0.8, i.e. not too high or too low, provided that test-retest and alternate form reliabilities are adequate.

Internal consistency reliability data is available for every version of OPQ, and is shown in the appropriate manuals.

2.8.3 Alternate Form Reliability

The extent to which two forms of a questionnaire measure the same dimensions of personality.

For two forms to be strictly equivalent they should not only seek to measure the same characteristic but also be based on the same item format.

Correlations between different versions of OPQ are not strictly alternate form studies, since they do not utilise the same item format.

2.8.4 Test-Retest Reliability

Correlates results on personality scales on one occasion with results for the same group at some later stage, usually at least one month later.

Concerned with the stability of a measure of personality over time.

If a questionnaire is a reliable instrument then the two sets of scores derived from the same person on two separate occasions should have a high degree of correspondence to each other. We know that although an individual’s personality is relatively stable, changes can be induced through associated situational or intra-personal factors. Changes in personality reflected by two measures over a given time interval may be due in part to changes within the individual, rather than purely the inconsistency of the questionnaire.

54

2.8.5 The Standard Error of Measurement (SEm)

The reliability coefficient (rt) can be used to produce the Standard Error of Measurement (SEm), which sets the band of error or tolerance that should be allowed for use when interpreting individual scale scores. Where the reliability is high, the SEm or band of error is small; however for an unreliable scale the band of error is correspondingly wider. The formula for the SEm is as follows:

SEm = SD

Where SD = standard deviation of the scale in raw scores or stens rt = reliability coefficient of the scale

It is applied as a plus and minus figure around scale scores, so forming a band of error.

The range of scores one SEm either side of a test result corresponds to a 68% confidence level.

If the measurement be repeated there is a 68% probability that it will fall within that band of error. If greater confidence is required then a band of error of plus and minus two SEms is applied to give a 95% level of confidence.

For OPQ scales plus and minus one sten corresponds to one Standard Error of Measurement either side of the scale score. Raw score and sten score SEms are provided for all versions of the OPQ.

This is based on the typical OPQ32 reliability coefficient of 0.75 and the Sten scale Standard Deviation of 2. A worked example looks like this:

SEm = 2

= 1 Sten

This is a “rule of thumb” for OPQ interpretation and can been seen as the error bars on OPQ profile charts.

55

Exercise 12

1. Describe how Internal Consistency is estimated.

2. Why might you argue that internal consistency coefficients in the range of 0.6-0.8 are the optimum for personality questionnaire scales?

3. If the reliability of a scale increases what happens to its SEm? You may find it useful to look at the formula for the SEm.

4. Which two criteria were applied to selecting good items for the OPQ?

5. What other final stages of research and development were necessary before releasing the questionnaire?

56

This is the end of the Precourse Work. In the next section some of the issues covered here will be looked at in more depth during the course, however, many will be taken as assumed knowledge and built upon so you should ensure that you are familiar with these ideas. The assessment at the end of the course may address materials covered in this booklet. If you have any questions, please call our Helpline.

The next section is revision of statistical issues.You may wish to look at it to refresh your memory on these ideas. If not, we look forward to seeing you on the course!

57

SECTION 3

Statistics Revision

(Optional)

58

3. STATISTICS REVISION GUIDE

A review of the statistics covered on the SHL Occupational Testing Course

Exercise 13The following questions review some important concepts in psychometric testing, including NORMS, CORRELATION and RELIABILITY. Work through these questions, referring to your Occupational Testing course notes where necessary, and then look to the back of the booklet to check your answers.

1. What does the standard deviation of a group of test scores tell us?

2. The following sets of scores have the same mean but which would have the larger standard deviation? (You should not need any calculation)

X Y

60 4070 6080 8090 100100 120

59

3 a) Calculate the Standard Deviation of the following set of raw scores.

Use the formula: SD = X X

N

2

Remember, you need to calculate the mean raw score first.

Raw Score (X-X) (X-X)232293230292830

b) If the range of these raw scores was restricted, how would the SD be affected?

60

4 Given that the group’s test scores are normally distributed:

a) What proportion of the group will fall between 1 SD below and 1SD above the mean?

b) What proportion of the group will fall between 2 SD below and 2 SD above the mean?

5 Draw the following distributions:

NORMAL POSITIVE SKEW NEGATIVE SKEW

6 What would a negatively skewed distribution suggest about the difficulty of a test?

61

7 Complete the following table:

PERCENTILE Z-SCORE T-SCORE STEN

Definition The proportion of the comparison group that an individual has scored better than

A standard score band system which divides the score scale into 10 divisions (standard ten)

Range -3.0 to +3.0

Derivation Z score x 10 + 50

SD 2.0

Mean 5.5

8 Which of the above are standard score systems?

9 What are the advantages of standard score systems?

62

10 Look at the following scattergram:

Test Scores - Applicants for Clerical Posts in a Building Society

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7

Test A

Test

B

If a candidate has a high score on Test B is their score likely to be high or low on Test A?

63

11 The correlation co-efficient is the statistic we most often use to measure the relationship between variables. It summarises the relationship as a single index.

a) What is the range of a correlation co-efficient?

___________ to ___________

b) What does the size of a correlation co-efficient tell us?

c) What does the sign (+ or -) of a correlation coefficient tell us?

d) If you have a correlation of 0 (zero) between two tests, what does this mean?

64

12 Give a brief description of the three main methods used to obtain reliability estimates:

a) Test-retest reliability:

b) Alternate form reliability:

c) Internal consistency:

13 The Standard Error of Measurement (SEm) is an estimate of the band of error that we need to leave round an individual test score in order to take account of errors of measurement. As the reliability of a test increases, what will happen to its SEm?

14 A particular verbal reasoning test has an SEm of 2 raw scores. If a candidate obtains a raw score of 24 on this test, in what range of scores is the true score likely to fall?

a) 68% confidence

b) 96% confidence

15 When selecting a norm group for a group of applicants, what factors would you consider to get the “closest match”?

65

ANSWERS TO EXERCISES4. ANSWERS TO THE EXERCISES

1 What does the standard deviation of a group of test scores tell us?

The standard deviation gives an indication of the degree of dispersion of the scores from the mean and an estimate of the variability of results in the total sample.

2 The following sets of scores have the same mean but which would have the larger standard deviation? (You should not need any calculation)

X Y

60 4070 6080 80 Y would have the largest SD90 100100 120

3 a) Calculate the Standard Deviation of the following set of raw scores.

Use the formula: SD = å (X X)2 N

Remember, you need to calculate the mean raw score first.

Raw Score (X-X) (X-X)232 +2 429 -1 132 +2 430 0 029 -1 128 -2 430 0 0

SD =2

b) If the range of these raw scores was restricted, how would the SD be affected?

The SD would be lowered

66

4 Given that a group’s test scores are normally distributed,

a) What proportion of the group will fall between 1 SD below and 1 SD above the mean?

68%

b) What proportion of the group will fall between 2 SD below and 2 SD above the mean?

95%

5 Draw the following distributions:

NORMAL POSITIVE SKEW NEGATIVE SKEW

6 What would a negatively skewed distribution suggest about the difficulty of a test?

Too easy .... most people do well ... distribution skewed to the right hand side

7 Complete the following table:

PERCENTILE Z-SCORE T-SCORE STEN

Definition

The proportion of the comparison group that an individual has scored better than

Indicates how many standard deviations above or below the mean a score is

A transformed Z-score. It gets rid of the confusion associated with negative scores and decimals

A standard score band system which divides the score scale into 10 divisions (standard ten)

Range 1%ile to 99%ile -3.0 to +3.0 20 to 80 1 to 10

Formula X-XSD

Z score x 10 + 50

Z score x 2 + 5.5

SD 1 10 2.0

Mean 0 50 5.5

67

8 Which of the above are standard score systems?

Z-scoresT-scoresStens

9 What are the advantages of standard score systems?

They enable the user to evaluate whether the individual has scored above or below average, and how far away they are from the average. The beauty of standard scores is that they transform scores onto a linear scale which can be mathematically manipulated, unlike percentiles.

10 Look at the following scattergram:

Test Scores - Applicants for Clerical Posts in a Building Society

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7

Test A

Test

B

If a candidate has a high score on Test B is their score likely to be high or low on Test A?

High

68

11 The correlation co-efficient is the statistic we most often use to measure the relationship between variables. It summarises the relationship as a single index.

a) What is the range of a correlation co-efficient?

-1.0 to +1.0

b) What does the size of a correlation co-efficient tell us?

The strength of the relationship between the two variables. The nearer to 1 the stronger the correlation.

c) What does the sign (+ or -) of a correlation coefficient tell us?

The direction of the relationship between the two variables

d) If you have a correlation of 0 (zero) between two tests, what does this mean?

There is no clear relationship between the two variables

12 Give a brief description of the three main methods used to obtain reliability estimates:

a) Test-retest reliability: Coefficient obtained by administering the same test to the same people on two separate occasions. Test scores from the two occasions are then correlated.

b) Alternate form reliability: Coefficient is obtained by administering two alternate (or parallel) forms of the same test to the same group of people. Test scores for the separate forms are then correlated.

c) Internal consistency: Coefficient is obtained by measuring the consistency of performance on all items in a test. This can be done by correlating the scores on all the odd items with those on the even items. More complex statistical procedures for (split half method) producing internal consistency estimates include Kuder Richardson and Cronbach’s Alpha techniques.

13 The Standard Error of Measurement (SEm) is an estimate of the band of error that we need to leave round an individual test score in order to take account of errors of measurement. As the reliability of a test increases, what will happen to its SEm?

The SEm will decrease.

69

14 A particular verbal reasoning test has an SEm of 2 raw scores. If a candidate obtains a raw score of 24 on this test, in what range of scores is the true score likely to fall?

a) 68% confidence

b) 96% confidence

15 When selecting a norm group for a group of applicants, what factors would you consider to get the “closest match”?

Work experience, education, industry sector, job type, age range, sample

70