Administration
Manual
Managers ♦♦♦♦♦ Graduates
KCP Potentia Series
KCPMaking Psychology Accessible
Potentia
Tests
Ability
Page 2KCP Potentia Series
KCP Potentia Series Ability Test Manual
by
Andrew Roberts BSc CPsychol
Knight Chapman Psychological Ltd
1 High Street
Lewes
East Sussex BN7 2AD
Page 3KCP Potentia Series
KCP Potentia Series Ability Test Manual
ACKNOWLEDGEMENTS
We have been extremely fortunate to have had considerable assistance in the developing,
trialling, validation and design of the Series.
We would like to extend our special thanks and gratitude to:
Rachel Frost of KCP who assisted in the development of this battery.
Alison Cox of KCP, who contributed extensively to the original design, development and
scoring of the tests, as well as to the necessary but exhausting task of compiling trialling
materials.
Leigh Grainger of the University of Cardiff's Department of Psychology for her creative input
into the development of the Reference Materials and test items, and for her considerable
help in trialling the tests.
Dr Chris Miles of the University of Cardiff for arranging large numbers of “guinea-pigs” for
the initial trialling.
Janice Truscott and her colleagues at Cadburys for all their help in trialling and validation.
Kyrstie Broadhead and her colleagues at Tesco for assistance in trialling.
Peter Harris and his colleagues at Llanelli Borough Council for assistance in trialling and
validation.
Peter Messer of Peter Messer Design Consultants in Lewes for the design of the Reference
Materials.
Keith Gillies of Caburn Communications for the original design of KCP test materials.
Paul Dale and John Cooper of the Venture Press, Scaynes Hill, West Sussex for all the
proofing and printing.
Page 4KCP Potentia Series
KCP Potentia Series Ability Test Manual
CONTENTS PAGE
BACKGROUND AND DEVELOPMENTAL OBJECTIVES 5
DESCRIPTION OF THE TESTS 6
ABILITY TEST ADMINISTRATION 8
Reference Material 10
Verbal Reasoning 11-12
Numerical Reasoning 13-14
Creative Problem Solving 15-16
ABILITY TEST SCORING 17
Verbal & Numerical Reasoning 17
Creative Problem Solving 18-28
Norm Tables 29
RELIABILITY 31-33
INTERCORRELATIONS 34
VALIDITY 35-36
Page 5KCP Potentia Series
KCP Potentia Series Ability Test Manual
BACKGROUND AND DEVELOPMENTAL OBJECTIVES
The Potentia Series was developed in response to the growing need for an integrated high
level assessment package.
Prior to Potentia there were several aptitude tests for graduates and managers on the
market, and in addition there were assessment exercises suitable for selection and
development at this level.
At Knight Chapman Psychological, however we became aware that assessors were rarely in
the situation where they found assessment exercises suited to their precise requirements. As
a result, many assessment centres have been designed on the basis of materials that
happened to be available “off-the-shelf”.
Candidates have therefore been presented with a disjointed and unrealistic procedure where
their abilities and working style have been assessed across discrete, unrelated scenarios -
one minute they are expected to market a cosmetic, the next they find themselves
manufacturing kitchen appliances.
In reality, of course, managers and professionals very rarely resolve problems which are so
disjointed and lack interdependence.
The Potentia package is designed to provide a complete set of associated, realistic
assessment materials to reliably measure a wide range of core management and graduate
competencies. The individual tests and exercises can be administered separately or in any
combination.
The structure of the series is such that all tasks relate to a common set of reference materials
which reflects a realistic but neutral work situation. Candidates will immediately realise the
relevance of the tests to selection or development within a graduate/managerial environment.
Gaining their ‘buy in’ is accepted as crucial to reliable assessment.
Page 6KCP Potentia Series
KCP Potentia Series Ability Test Manual
DESCRIPTION OF THE TESTS
Structure
The Potentia Series consists of a number of tests and exercises which measure a range of
high level problem solving abilities and skills recognised to be important for effective
managerial and graduate trainee positions.
ABILITY TESTS
The tests which are currently available to Potentia users are:
Verbal Critical Reasoning
Numerical Critical Reasoning
Creative Problem Solving
The questions in each of the tests relate to a set of documents, which candidates have 10
minutes to look through at the beginning of the Potentia administration session. There are 11
documents which refer to the proposed takeover of one small theatre by a larger group of
theatres. The documents create a general scenario, as well as providing specific verbal and
numerical information.
All of the test themselves can be administered individually or in any combination.
Verbal Critical Reasoning
Each question consists of a statement which the candidate must evaluate in the light of the
information given in the 11 documents. A three-way multiple-choice format requires the
candidate to decide whether the statement is true, or false, or whether there is insufficient
information given in the inserts to say for certain whether the statement is true or false.
Numerical Critical Reasoning
The candidate is required to use data contained in the 11 documents to answer numerical
critical reasoning questions. Although arithmetic calculations (addition, subtraction,
multiplication, division) and basic algebra are required, the emphasis is on reasoning rather
than arithmetic processing. Numerical concepts including graph reading, ratios, table
reading, percentages, percentage change, and demographic comparisons are
featured. Face and content validity are enhanced by the use of data and concepts which
candidates will recognise as relevant to real-life situations.
Page 7KCP Potentia Series
KCP Potentia Series Ability Test Manual
Creative Problem Solving
The candidate is required to list possible causes or reasons for given events. These events
are within the context of the information contained in the 11 documents. Candidates are
informed that this is a test of creativity and are encouraged to use their imagination in trying to
think of unlikely causes as well as more probable ones. Candidates are also asked to try to
include some causes which other people will probably not have thought of.
Page 8KCP Potentia Series
KCP Potentia Series Ability Test Manual
ADMINISTRATION
Research has shown that small differences in the way tests are administered have significant
effects on scores resulting from the tests. The administration of tests is therefore critical to
the provision of reliable, uncontaminated measurements. Great care must be taken to ensure
standardisation.
Preparation
The objective is to run a well coordinated testing session with no emergencies. Test
administrators should make sure they are completely familiar with the instructions and test
procedure. They should also fully understand the test objectives and background so that they
are able to answer questions raised by test takers.
Materials should be carefully counted and checked. Administrators should ensure that
pencils, rubbers, rough paper, pocket calculators, test booklets, Reference Material and
answer sheets are all collated and that a stop watch, administration booklet, test manual
and scoring key are to hand.
The test conditions also need some consideration prior to the testing session. A suitable
room should be chosen which will be free from undue noise and distraction, and will
provide adequate lighting, ventilation/heating, seating and space. The test administrator
should take steps to ensure that there are no interruptions during the session: telephones
should be disconnected or redirected and a sign should be posted on the door outside
indicating that testing is in progress. Other personnel should be briefed that interruptions
are strictly prohibited.
The administrator’s delivery of the introduction should be such that it reassures,
encourages and motivates the candidates.
The introduction should include details of the timetable, domestic arrangements (including
where the lavatories are located) and a description of the test to be taken. The purposes
of the test should be “sold” to the candidates - in particular, it should be pointed out that a
good fit between the job requirements and the job holder is in everybody’s interests.
Introduction
Prior to the formal test instructions, a less formal introduction should be delivered to the
candidates. The purpose of this is partly to provide relevant information, but also to build
rapport.
Rapport, in this context, refers to the administrator’s efforts to arouse the candidates
interest in the test, elicit their cooperation, and encourage them to give their full
concentration. The content of the introduction should eliminate the element of surprise
from the situation. Uncertainty is a major component of test anxiety and so should be
reduced as far as possible before the testing begins.
Page 9KCP Potentia Series
KCP Potentia Series Ability Test Manual
Test Instructions
It is imperative that these are presented in a standardised way according to the
administration booklet. In order to ensure that every candidate receives the same
instructions, these should always be read, rather than memorised.
The instructions which appear in the administration booklet are given on the following
pages.
Timing
Clear start and stop instructions should be given by the administrator. The test durations must
be timed exactly using a stopwatch. A secondary watch or clock should also be used in case
the stopwatch goes wrong or a candidate disputes the time given.
The time allowed for each test is:
* Reference Material 10 minutes
* Verbal Critical Reasoning 16 minutes
* Numerical Critical Reasoning 35 minutes
* Creative Problem Solving 14 minutes
Sequence of Tests
The Reference Material is always administered first. Subsequent tests can be administered
in any order, indeed, there may be occasions where only one test, for example, Numerical
Reasoning is being used.
However, if the entire battery is to be administered, we would generally use the sequence of
Verbal Reasoning, Numerical Reasoning and then Creative Problem Solving.
Page 10KCP Potentia Series
KCP Potentia Series Ability Test Manual
POTENTIA - REFERENCE MATERIAL ADMINISTRATION INSTRUCTIONS
Explain to the candidates which tests they will be asked to take and outline the timetable for the
session. Provide any other background information which is appropriate (e.g. relevance of tests to
job content).
Ensure that candidates are comfortable and answer any questions they wish to raise.
Give each candidate:
• a Reference Material booklet
• 2 sheets of rough paper
• 2 pencils
• an eraser
Say:
“Please turn to the first page of your Reference Material Booklet, where you will find the instructions. Please
follow these instructions whilst I read them.
The questions in each of the tests relate to this Reference Material which you will have 10 minutes to look
through before you begin the tests themselves.
This Reference Material consists of eleven documents relating to an ailing theatre company which is being
threatened with takeover by a large organisation, the Royal Theatre Company.
The documents have been put together for the Chairman of this group.
Before you start each test you will receive full instructions explaining what you have to do.
During the next 10 minutes you may not have time to read all of the eleven documents in detail, but try to get a
feel for the kind of information contained in each.
Remember, you will have this material to refer to throughout the tests. If you would like to make notes, you
may use the rough paper provided but please do not write on the Reference Material Booklet or Questionnaire
Booklets.
Are there any questions at this stage?”
Pause. Deal with any questions raised.
Ask the candidates to:
“Turn to the next page of your Reference Material booklet and start looking through the reference material
NOW.”
After EXACTLY 10 minutes say:
"STOP. Please close your Reference Material booklet now."
Check that all candidates have closed their Reference Material booklets.
Test administrators should now refer to the specific administration instructions for each of the tests to
be taken.
Page 11KCP Potentia Series
KCP Potentia Series Ability Test Manual
POTENTIA - VERBAL REASONING ADMINISTRATION INSTRUCTIONS
Say: “This is a test of your verbal critical reasoning ability - your ability to identify logic in written argument.
Each of the questions in this test consists of a statement which you must evaluate in the light of the
information given in the Reference Material about the Heritage Theatre.
Give your answers on the Answer Sheet by filling in circle A, B or C according to these rules:
Fill in Circle A if the statement is TRUE or follows logically from the information given.
Fill in Circle B if the statement is FALSE from the information given in the Reference Material.
Fill in Circle C if there is INSUFFICIENT INFORMATION given in the Reference Material to say for certain
whether the statement is true or false.
These rules are given on each page of the test.
Tell the candidates not to open their test booklets until asked to do so. Provide the candidates with
any background information about the test and its purposes. Ensure that the candidates are
comfortable and answer any questions they wish to raise.
At the beginning of the test administration session, candidates should have spent 10 minutes looking
through the Reference Material (see Potentia Reference Material Administration Instructions).
Say: “ Please write your full name and today’s date in the spaces provided on your answer sheet.” Pause.
Say: ”Open your test booklet to the first page, where you will find the test instructions. Please follow these
instructions while I read them.” Pause.
Ensure that each candidate has:
• a PVT booklet
• a PVT answer Sheet
• a Reference Material Booklet
• a sheet of rough paper
• 2 pencils
• an eraser
When giving your answers, use ONLY the information given in the Reference Material. You should assume that
this information is correct even if it contradicts what you believe to be the case in reality.
Before you start the test there are some example questions. Please do these now in your own time, giving your
answers in the Example Section on the Answer Sheet.
You WILL need to refer to the Reference Material. When you have finished the three example questions,
please close your Reference Material booklet."
Check that the candidates are answering the example questions correctly, providing help where
necessary. When all of the candidates have completed the example questions correctly, say:
Page 12KCP Potentia Series
KCP Potentia Series Ability Test Manual
"The answer to Example 1 is C. There is Insufficient Information to say for certain whether the statement is
true or false.
The answer to Example 2 is A. The statement is True from the information given in Document 3.
The answer to Example 3 is A. The statement is True from the information given in the Entertainments
Research Company report - Document 9.
Are there any questions about these examples?" Pause. Deal with any questions raised.
POTENTIA - VERBAL REASONING ADMINISTRATION INSTRUCTIONS (cont'd)
Remember, you are allowed to use the Reference Material throughout the test.
Are there any final questions?"
Pause. Deal with any questions raised.
Ask the candidates to:
"Turn over to page 1 and BEGIN."
START THE STOPWATCH
During the test, the test administrator may answer any questions relating to procedure, but should
not help candidates with the questions themselves.
Note any significant problems or questions raised by candidates.
After EXACTLY 16 minutes say:
"STOP. Please close your Reference Material Booklet and your Test Booklet."
If you are going on to administer further tests, collect in the materials which are no longer required,
i.e. answer sheets and test booklets, but ensure that the candidates retain their Reference Material
booklets, pencils, erasers and rough paper. If you are not administering additional tests, collect all
materials and thank the candidates for doing the test.
Say: "Please note the following points.
You will have sixteen minutes for the thirty questions in this test. Work as quickly and accurately as you can.
If you are not sure of an answer, move on and return to it later rather than simply guessing. Please use the
pencils provided. Rub out and change your answers if you wish.
Page 13KCP Potentia Series
KCP Potentia Series Ability Test Manual
POTENTIA - NUMERICAL REASONING ADMINISTRATION INSTRUCTIONS
Ensure that each candidate has:
• a PNT booklet
• a PNT answer sheet
• a pocket calculator
• 2 sheets of rough paper
• a Reference Material booklet
• 2 pencils
• an eraser
Tell the candidates not to open their test booklets until asked to do so. Provide the candidates with
any background information about the test and its purposes. Ensure that the candidates are comfort-
able and answer any questions they wish to raise.
At the beginning of the test administration session, candidates should have spent 10 minutes looking
through the Reference Material (see Potentia Reference Material Administration Instructions).
Say: “Please write your full name and today’s date in the spaces provided on your answer sheet.” Pause.
Say: ”Open your test booklet to the first page where you will find the test instructions. Please follow these
instructions while I read them.” Pause.
Say: “This is a test of your numerical critical reasoning ability- your ability to manipulate and integrate work-
related data. Each of the questions in this test relates to data contained in the Reference Material. For each
question you are given five answers to choose from. Only one answer is correct in each case. Decimals are
generally rounded to two decimal places.
Please note that some questions may be impossible to answer from the data that you have been given. In
these cases “Insufficient data” is the correct answer. You may use a pocket calculator and the rough paper
provided.
Give your answer by filling in the appropriate circle A,B,C,D or E on the Answer Sheet.
Before you start the test there are some example questions. Please do these now in your own time, giving your
answers in the Example Section on the Answer Sheet.
You WILL need to refer to the Reference Material. When you have finished the three example questions,
please close your Reference Material booklet.”
Check that the candidates are answering the example questions correctly, providing help where
necessary. When all of the candidates have completed the example questions correctly, say:
“The answer to Example 1 is A - 65.80%
The answer to Example 2 is D - 1,019
The answer to Example 3 is E - Insufficient data.
Are there any questions about these examples?”
Pause. Deal with any questions raised.
Page 14KCP Potentia Series
KCP Potentia Series Ability Test Manual
Say: “Please note the following points:
You will have 35 minutes for the 26 questions in this test, so you will need to work quickly but accurately.
Use the pencils provided to fill in your answers on the Answer Sheet so that you can rub out and change your
answers if you wish. If you are not sure of an answer, move on and return to it later rather than simply guessing.
Remember, you may refer to the Reference Material throughout the test.
Are there any final questions?”
Pause. Deal with any questions raised.
Ask the candidates to:
“Turn over to page 1 and BEGIN.”
START THE STOPWATCH
During the test, the test administrator may answer any questions relating to procedure, but should not
help candidates with the questions themselves.
Note any significant problems or questions raised by the candidates.
After EXACTLY 35 minutes say:
“STOP. Please close your Reference Material booklet and your Test booklet.”
If you are going on to administer further tests, collect in the materials which are no longer required,
i.e. answer sheets, test booklets and calculators, but ensure that the candidates retain their
Reference Material booklets, pencils, erasers and rough paper. If you are not administering
additional tests, collect all materials and thank the candidate for doing the test.
POTENTIA - NUMERICAL REASONING ADMINISTRATION INSTRUCTIONS (cont'd)
Page 15KCP Potentia Series
KCP Potentia Series Ability Test Manual
POTENTIA - CREATIVE PROBLEM SOLVING ADMINISTRATION INSTRUCTIONS
Ensure that each candidate has:
• a Creative Problem Solving test booklet
• a Reference Material booklet
• 2 pencils
• a rubber
Tell the candidates not to open the test booklet until told to do so.
At the beginning of the test administration session, candidates should have spent 10
minutes looking through the Reference Material. (See Potentia Reference Material
Administration Instructions.)
Ask the candidate to :
“Please write your full name and today’s date in the spaces provide on the front cover of the
test booklet.”
Pause
“Open the test booklet to page 1 where you will find the test instructions.”
Pause
“Please follow these instructions whilst I read them.
This is a test of your creative problem solving ability - your ability to generate ideas and be
imaginative.
Each of the questions in this test consists of a possible problem which is broadly related to
the scenario contained in the Reference Material about the Heritage Theatre.
There are four problems.
Your task is to produce a list of the possible causes of each of the given problems. Try to
be imaginative. Write down as many ideas as you can. There are no right or wrong
answers, so try to come up with unusual or unlikely ideas which other people may not have
thought of, as well as more probable explanations.
Page 16KCP Potentia Series
KCP Potentia Series Ability Test Manual
Please remember:
Write as clearly as you can.
Use the spaces provided on the following pages to write your lists.
If you run out of space on a page, turn over and use the blank paper on the next side.
Make sure that your answers are relevant to the problem.
You will have fourteen minutes for the four problems in this test, so you will need to work
quickly.”
Say:
“Remember, you may refer to the Reference Material during the test.
Are there any final questions?”
Pause. Deal with any questions raised.
Ask the candidates to:
“Turn over to page 2 and BEGIN.”
START THE STOPWATCH
During the test, the test administrator may answer any questions relating to proce-
dure, but should not help candidates with the lists themselves.
Note any significant problems or questions raised by the candidates.
After EXACTLY 14 minutes say:
“STOP. Please close your Reference Material booklet and your Test booklet.”
If you are going on to administer further tests, collect in the materials which are no
longer required, i.e. the test booklet, but ensure that the candidates retain their
Reference Material booklets, pencils and rubbers. If you are not administering
additional tests, collect all materials and thank the candidates for doing the test.
Page 17KCP Potentia Series
KCP Potentia Series Ability Test Manual
SCORING - VERBAL AND NUMERICAL REASONING
Verbal Reasoning and Numerical Reasoning are both multiple choice tests with systematic
scoring systems. Scoring keys are used to score the total number of correct answers to
produce a raw score which is then interpreted by comparison with a norm group. The stages
in scoring are:
1. Check the answer sheet before using the scoring key. If the candidate has filled
in more than one box for a particular question, no marks are given for this question.
2. Align the scoring key so that the circles on the answer sheet show through the
reference circles on the Key.
3. Count the number of questions answered correctly. Write the total number of
correct answers in the box marked RS on the answer sheet.
4. Count up the total number of items attempted, and write this in the box marked RS
on the answer sheet as well.
5. Choose an appropriate norm table. Make a note of the table used in the box
marked ‘comparison group.’
6. Use the norm table to convert the raw score to a percentile score or standard
score.
Page 18KCP Potentia Series
KCP Potentia Series Ability Test Manual
SCORING - CREATIVE PROBLEM SOLVING TEST
General Points
The Creative Problem Solving Test is designed to measure the candidate’s ability to produce
ideas creatively. This is separate and different from the candidate’s critical reasoning
abilities. Critical reasoning is a convergent thinking process - the Creative Problem Solving
Test is concerned with divergent thinking.
Assessors are often tempted to give better scores to responses which are logical or “based
on common sense”. However, since this is not a test of reasoning abilities or judgement, this
temptation should be resisted. Creative Problem Solving is purely a test of creative
production and responses should be assessed with this in mind.
Open ended tests are inevitably more difficult and less objective to score than multiple choice
tests. Assessors should aim to stick firmly to the guidelines which follow, but will need to
generalise from these guidelines on occasions. This is because the number of potential
responses to the questions is infinite and no set of scoring instructions could comprehensively
cover all eventualities.
Ideally two separate assessors should score each candidate’s responses independently and
then discuss and rectify any disparities between the generated scores.
It is strongly recommended that this approach of dual scoring is adopted when the Creative
Problem Solving Test is first implemented, as this process highlights areas where
standardisation can be tightened up.
There are three scores:-
* Fluency
* Lateral Flexibility
* Originality
The procedure for determining a score on each of these dimensions of creative thinking is
covered on the following pages.
Page 19KCP Potentia Series
KCP Potentia Series Ability Test Manual
Scoring Procedure
Fluency (Column One)
The fluency score is a measure of the quantity of ideas produced and is based on a simple
count of the acceptable responses given. For each item, count up the number of acceptable
responses and record the total in the box at the foot of the first column.
Responses should NOT be counted if they are:-
Totally irrelevant to the given fact or event.
Duplicates of previously given responses, i.e. a simple rewording of precisely the
same idea.
These criteria of unacceptability should be interpreted liberally. Unacceptable responses are
quite uncommon. If in doubt, assessors should allow responses as acceptable. However, if a
response is unacceptable it should be discounted from the scoring of originality and lateral
flexibility as well as fluency.
Lateral Flexibility (Column Two)
The Lateral Flexibility score concerns the range, or breadth of responses given. If one
candidate has given five responses to a particular item, but all five ideas are along similar
lines, the lateral flexibility score will be low. If another candidate has also given five responses
to the same item, but the five ideas are diverse in their content, the lateral flexibility score will
be higher.
For each test item, refer to the scoring guidelines on the following pages. These guidelines
indicate the headings into which responses may fall. Against each of the candidate’s ideas,
make a note of the heading by writing its letter, A B, C etc. in column two. If an idea is clearly
different and does not fit into one of the given categories, it should be given an “O” for “Other”.
Now add up the total number of different letters, and write this score in the box at the bottom
of column two. If there are “O” ratings, each one is awarded a mark.
Please note that for each heading, we have provided some examples. These are for
illustrative purposes only and by no means represent the total universe of responses that
might appear under each heading.
Page 20KCP Potentia Series
KCP Potentia Series Ability Test Manual
Originality (Column Three)
The score for originality is naturally more qualitative than quantitative. “Originality” in this
context is a measure of the rarity or unusualness of the responses given. Higher originality
scores are given for responses which are not common amongst other candidates. The
“commonness” or popularity of responses has been ascertained through content analysis of
trialling data. However, since the same kind of idea may be expressed in many different
ways, the originality of each response is assessed on the basis of the category that it falls
into.
When scoring an item for originality, also note the ‘Extra mark’ headings. Generally speaking,
an additional mark should be awarded if the response is fuller, more detailed and more
specific.
Responses which are highly unusual are placed in the “Other” category. These automatically
attract 3 marks, and no extra specificity marks should be given.
Our experience with the Supra Series suggests that “Other” categories are relatively few and
far between, and assessors should be sparing in their use of this category.
Page 21KCP Potentia Series
KCP Potentia Series Ability Test Manual
ITEM 1. International news reports suggest that theatres in the UK are a
thing of the past.
A. Economic/financial
e.g
People have less money to spend.
Theatres become more expensive.
Extra mark - Fuller explanation such as large scale redundancies mean
that more people are unemployed and therefore have less money to
spend.
Originality score - 0
B. Improvement/Increase in other entertainment options
e.g
Television more popular.
People prefer computer games.
Video take over.
Extra mark - Fuller explanation such as a new TV company is so popular
that no one leaves their own home for entertainment.
Originality score - 0
C. The standard of theatres is low/has declined
e.g
Theatres are dilapidated.
Props have fallen apart.
Extra mark - Fuller explanation such as the seats have all been removed
from theatres as part of the war effort.
Originality score - 0
D. Fashion trends/youth culture
e.g
Not ready/old enough to go to the theatre.
Young people are more interested in other things.
Extra mark - Fuller explanation such as schools no longer teach young
people to appreciate cultural entertainment.
Originality score - 1
Page 22KCP Potentia Series
KCP Potentia Series Ability Test Manual.
E. Advertising/marketing
e.g
Theatres not promoting themselves anymore.
Poor marketing and advertising.
Extra mark - Fuller explanation such as a particular marketing campaign
was badly handled and had the opposite effect to that intended, putting
people off from going to the theatre.
Originality score - 2
F. Incorrect information
e.g
Newspaper got figure wrong.
Journalists interpreted the statistics wrongly.
Extra mark - Fuller explanation such as the journalist is paid by the
government to distort figures to make the UK economy look better than it is.
Originality score - 2
O. Other
e.g
All UK actors and directors have fled the country and gone to Hollywood.
No extra marks.
Originality score - 3
Page 23KCP Potentia Series
KCP Potentia Series Ability Test Manual
ITEM 2. The Heritage Theatre wins its takeover battle and resists Inglebert.
A Aspects specifically mentioned already in the Reference Material
(excluding local community objections).
e.g
Marketing or financial ideas mentioned in the Reference Material.
Inglebert’s promotional ideas.
Extra mark - Where a candidate significantly builds upon the information
given in the Reference Material.
Originality score - O
B. Local community objections/support
e.g
This local community showed their support for the theatre by attending
productions.
Strong objections from the local community.
Extra mark - Fuller explanation such as the local community were so worried
about losing their theatre that they organised fund-raising events to save it.
Originality score - O
C. Inglebert/RTC go off the idea and withdraw
e.g
Inglebert follows the Entertainments Research Company’s recommendation and
withdraws.
The RTC Directors put pressure on Inglebert to withdraw bid.
Extra mark - Fuller explanation such as Inglebert is charged with fraud
and the RTC have to sell off assets to pay for his bail.
Originality score - O
D. Cash injection (other than from the local community)
e.g
Anonymous supporter bequeaths funds to the theatre.
Government gives grant aid.
Extra mark - Fuller explanation such as a local millionaire commits suicide
and has donated his whole estate to the theatre.
Originality score - 1
Page 24KCP Potentia Series
KCP Potentia Series Ability Test Manual
E. Entirely new marketing ideas
e.g
Mailshots are targeted better.
A celebrity visits the theatre to pledge support.
Extra mark - Fuller explanation such as a programme is sent out to all
social clubs offering a reduction in price if more than 10 individuals take
out an annual subscription.
Originality score - 2
F. The building is used for another purpose.
e.g
The Heritage is turned into a bingo hall.
The theatre is used for dances.
Extra mark - Fuller explanation such as a local company establishes a
contract to hire the theatre regularly in order to sell holiday time-shares.
Originality score - 2
G. Other
e.g
By coincidence, the theatre turns out to be the only venue suitable to stage a
play that is part of the core curriculum.
No extra marks.
Originality score - 3
Page 25KCP Potentia Series
KCP Potentia Series Ability Test Manual
ITEM 3. A marketing company has promised to increase the Heritage
Theatre’s sales threefold.
A. Advertising/marketing strategies already mentioned in the Reference
Material
e.g
Discounts given to old age pensioners.
Advertising space in programmes offered to local businesses.
Extra mark - Fuller explanation such as the Wednesday evening discount
is moved to Thursday as there is nothing worth watching on TV that night.
Originality score - O
B. Improvements in the quality of productions/reduction in ticket
prices
e.g
Ticket prices are slashed.
Better quality actors are employed.
Extra mark - Fuller explanation such as they hire big name performers
for random performances so that the audience takes pot luck as to
whether they get an amateur or a professional production.
Originality score - 0
C. Advertising/marketing campaign not mentioned in the Reference
Material
e.g
Advertising space bought on Tube trains.
Direct mail campaign targeted at specific sections of the community.
Extra mark - Fuller explanation such as tokens are given at local petrol
stations - collect 10 and get a free ticket at the petrol station’s expenses.
Originality score - 0
D. Increase money available for marketing or improvements to the
theatre
e.g
Marketing company finds a sponsor to invest in the theatre.
Marketing company invests money and takes a stake in the theatre.
Extra mark - Fuller explanation such as the marketing company has been
approached by an international philanthropist who wants to invest in the
theatre.
Originality score - 1
Page 26KCP Potentia Series
KCP Potentia Series Ability Test Manual
E. Use theatre for other purposes
e.g
Turn into a bingo hall.
Hire a “Chippendale” - type act.
Extra mark - Fuller explanation such as hire the theatre out for hen nights
starring local “Chippendales”.
Originality score - 2
O. Other
e.g
Pretend that the takeover bid was won by the Heritage and capitalise on
advertising the theatre as better and more successful than the RTC.
Originality score - 3
Page 27KCP Potentia Series
KCP Potentia Series Ability Test Manual
ITEM 4. Large groups are demonstrating outside the Heritage Theatre.
A. Takeover bid (excluding refurbishment)
e.g
Local community demonstrating against (or for) takeover.
Actors lobbying for support.
Extra mark - Fuller explanation such as English National Heritage
organises a mass demonstration to show support for independent
theatres which are being forced to close.
Originality score - O
B. Refurbishment
e.g
Heritage losing its Victorian facade.
Closure of theatre due to refurbishment.
Extra mark - Fuller explanation such as the “Save Victorian Buildings Society”
organise a mass demonstration to protest against the loss of Victorian values.
Originality score - O
C. Contentious play/performance
e.g
Sexist play.
Script is considered racist.
Extra mark - Fuller explanation such as an anti-drug campaign against
“Needles and Opium” portraying drug use as glamorous.
Originality score - 1
D. Demonstration which has nothing to do with theatres
e.g
Animal rights activists.
Dock workers have come out on strike.
Extra mark - Fuller explanation such as a strike has been called by
dockers because a property developer has bought the site and plans to
turn it into an exclusive shopping centre.
Originality score - 2
Page 28KCP Potentia Series
KCP Potentia Series Ability Test Manual
E. Prices too high
e.g
People resent ticket price increases.
Discounts withdrawn.
Extra mark - Fuller explanation such as Winthorpe has doubled the ticket
prices in order to raise money for refurbishing the interior of the theatre.
Originality score - 2
O. Other
e.g
Demonstration in support of an actor who is accused of an assassination
attempt on the Queen.
No extra marks.
Originality score - 3
Page 29KCP Potentia Series
KCP Potentia Series Ability Test Manual
PERCENTILE NORM TABLES
Composite graduate group (N = 1354, Verbal and Numerical
N= 90, Creative Problem Solving)
%ile Verbal Numerical Fluency Lateral Flex Originality %ile
99 23-26 36+ 21+ 33+ 99
97 28-30 21-22 33-35 19-20 30-32 97
95 26-27 19-20 30-32 18 27-29 95
90 24-25 17-18 28-29 16-17 24-26 90
85 23 16 26-27 15 22-23 85
80 22 15 24-25 20-21 80
75 21 14 23 14 19 75
70 20 13 22 13 18 70
65 19 12 20-21 16-17 65
60 18 11 19 12 15 60
55 17 10 18 11 14 55
50 16 17 13 50
45 15 9 16 10 11-12 45
40 14 8 15 10 40
35 13 7 14 9 9 35
30 12 6 12-13 8 8 30
25 11 5 11 6-7 25
20 10 4 9-10 7 5 20
15 9 3 7-8 6 2-4 15
10 7-8 1-2 5-6 4-5 0-1 10
5 5-6 0 2-4 3 5
3 2-4 0-1 2 3
1 0-1 0 0-1 1
Page 30KCP Potentia Series
KCP Potentia Series Ability Test Manual
T-SCORE NORM TABLES
Composite graduate group (N = 1354, Verbal and Numerical)
T Verbal Numerical
80
78
76 25-26
74 24
72 23
70 29-30 22
68 28 20-21
66 27 19
64 25-26 18
62 24 17
60 22-23 16
58 21 14-15
56 20 13
54 18-19 12
52 17 11
50 16 9-10
48 14-15 8
46 13 7
44 12 6
42 10-11 5
40 9 3-4
38 8 2
36 6-7 1
34 5 0
32 4
30 2-3
28 1
26 0
Page 31KCP Potentia Series
KCP Potentia Series Ability Test Manual
RELIABILITY
It is critical that tests are administered and scored in a standardised way, to avoid variability
creeping into the testing process and contaminating test results. However, even when
guidelines are adhered to, psychological tests cannot measure abilities with 100% accuracy.
What is Test Reliability?
Test reliability refers to the test’s degree of precision, or accuracy of measurement. To
understand this concept, we can use the analogy of measuring distance (or “length”). If we
wanted to measure the length of a football pitch, we could pace from one end to the other
counting each pace as roughly 1 metre. This would provide us with an imprecise indication of
the football pitch’s length.
A more reliable (or precise) method would be to use a tape measure. However, a tape
measure is not the most reliable instrument for measuring a length or distance: very precise
devices such as the micrometer can be used to measure the nearest thousandth of a
millimetre. So, “pacing”, the tape measure and the micrometer are three methods of
measuring distance which differ in their reliabilities. To measure a football pitch it would be
absurd to use a micrometer, but “pacing” may be too inaccurate: the level of reliability offered
by the tape measure method may make it the most appropriate option.
Like the “pacing” and tape measure methods, testing has a degree of inherent unreliability. In
evaluating tests for use in assessment, we are concerned to ensure that the test’s reliability is
acceptable. If scores are too inaccurate, wrong decisions will be made.
The reliability of a test is expressed as a correlation coefficient and is called a reliability
coefficient.
A reliability coefficient of 0.0 means a test has zero reliability; it is completely inaccurate and
the characteristic is being measured no more precisely than a random guess. A reliability of
1.0 means that a test is 100% reliable; it measures the characteristic with perfect precision.
However, this degree of reliability is never actually found. A reliability coefficient is thus an
index with a theoretical range of 0 to 1. In practice, reliabilities generally vary between 0.5
and 0.9.
Page 32KCP Potentia Series
KCP Potentia Series Ability Test Manual
Different Kinds of Reliability
Test-Retest Reliability
The test-retest reliability coefficient is the correlation between the scores obtained by the
same person on two separate administrations of the test. If test results are generally
different on the second administration from the first, the test-retest reliability coefficient will
be low. If there is a good relationship between scores on the two occasions, the
coefficient will be high.
Alternate-form Reliability
This is computed by comparing scores obtained by the same individuals on two forms (or
versions) of the same test. Of course, this method is limited to tests for which more than
one form exist.
Internal Consistency
Only one administration of the test is required to compute the internal consistency coeffi-
cient. This is derived by splitting the test questions into two and then correlating the
respective totals for the two halves. One approach is to divide out the odd and even
questions and then correlate odds with evens. More sophisticated ways of dividing the
questions into two sets are also available. If all of the questions are measuring the same
characteristic, the internal consistency coefficient should be high.
Page 33KCP Potentia Series
KCP Potentia Series Ability Test Manual
Internal Consistencies
Test N Mean SD Internal SEm
Consistency*
Verbal Reasoning 1354 15.93 6.75 0.85 2.61
Numerical Reasoning 1354 9.58 6.10 0.86 2.28
C.P.S: Fluency 90 14.68 8.19 0.77 3.92
C.P.S: Originality 90 10.57 9.37 0.72 4.96
C.P.S: Lat.Flex. 90 8.97 4.07 0.66 2.37
* Cronbach’s alpha
These internal consistencies suggest that the tests are reliable measures. The slightly lower
alpha for lateral flexibility is not unexpected. Creativity, by definition, cannot be measured with
the same degree of precision as numerical reasoning, for example. The initial categories that
we have established are open to some interpretation by scorers, and this is likely to lead to
some variability in scores.
Page 34KCP Potentia Series
KCP Potentia Series Ability Test Manual
INTERCORRELATIONS
Test scores from a composite group of undergraduates, graduates and managers were used
to compute the following intercorrelation matrix for the tests Verbal Reasoning, Numerical
Reasoning and Creative Problem Solving.
Verbal Numerical C.P.S. C.P.S. C.P.S.
Reasoning Reasoning Fluency Originality Lat.Flex
Verbal
Reasoning 1.00 0.58 0.17 0.15 0.18
Numerical
Reasoning 0.58 1.00 0.38 0.28 0.37
C.P.S
Fluency 0.17 0.38 1.00 0.68 0.85
C.P.S.
Originality 0.15 0.28 0.68 1.00 0.74
C.P.S.
Lat.Flex 0.18 0.37 0.85 0.74 1.00
The intercorrelation of 0.58 between Verbal Reasoning and Numerical Reasoning indicates a
moderate relationship which suggests that there is a connection between these forms of
critical reasoning ability. However the relationship is sufficiently weak to indicate that the two
tests are still measuring different abilities.
There are stronger relationships between the Creative Problem Solving scores resulting from
the three different scoring strategies. In particular, there is a very strong relationship (0.85)
between Fluency and Lateral Flexibility in spite of a fair degree of independence in the
scoring strategies themselves. The strength of the relationships between Fluency and the
other two scores cannot be fully accounted for by the fact that a greater number of responses
will mean a greater number of different categories represented. The high intercorrelations
indicate that the capacity for producing original ideas is linked to the individual's fluency in
producing ideas generally, and that lateral flexibility is related to both of these.
It may be concluded that the scores together are providing an overall measure of creative
production, in the same way that test scores form a range of reasoning tests taken together
may be said to provide an indication of general intelligence.
Page 35KCP Potentia Series
KCP Potentia Series Ability Test Manual
VALIDITY
The validity of a test is the degree to which it measures what it is supposed to measure. In
occupational settings therefore, we are generally interested in whether a particular test is
capable of measuring some attribute which we know to be relevant to effective job
performance.
Face validity
This form of validity is to do with appearances. Managers and graduates often complain that
a test that they have undertaken doesn't seem to represent the real world. This is most often
the case when they have been confronted with a test which consists of discrete, unrelated
items. In these circumstances, candidates may feel less happy about the relevance of the test
results.
Potentia, like its pioneering relative the Supra Series, has been carefully designed with face
validity in mind. The tests are based on a scenario which has a realistic feel. The scenario,
however, is relatively neutral. That is, we deliberately avoided producing tests based on
information which might in some way put some candidates at an advantage or a
disadvantage.
Content validity
A test is said to have content validity if the items are representative of the attributes to be
measured. With Potentia, we drew upon our experiences of over 20 years in assessing
graduates and managers, consciously writing items were in line with job analytic studies
which revealed the kind of abilities that graduates and managers were asked to employ.
Concurrent validity
This is the form of test validity which is most frequently reported by test publishers. It is
concerned with collecting test and job performance data (often called a "criterion measure")
at the same time. The relationship between the two is calculated to see whether, for example,
people who score better on the test also gain better performance ratings.
With Potentia, we conducted a study with a sample of managers in a local authority. These
managers came from a broad range of disciplines, from personnel through to technical
functions, and ranged from junior to senior management. Each manager completed Verbal
Reasoning, Numerical Reasoning and Creative Problem Solving. Independently, ratings of
their job performance were obtained.
The results were very encouraging. Managers' overall performance was correlated at the 5%
per cent level of statistical significance (that is, we can be 95% confident that the relationship
has not occurred by chance) for Numerical Reasoning, Fluency and Lateral Flexibility, and at
the 10% level for Verbal Reasoning.
Page 36KCP Potentia Series
KCP Potentia Series Ability Test Manual
These relationships are likely to be an underestimate of the tests true validity given that the
sample was restricted in range. We have not corrected for this restriction since the sample
population made up a fair proportion of the initial norm group.
Additional studies of construct and concurrent validity are on-going and users are encouraged
to let KCP know if they can assist with similar studies.
Equal opportunities
KCP is committed to an on-going research programme in the area of equal opportunity.
During the trialling of Potentia we were able to collect data from bother sexes and from ethnic
minority populations. However, this data were not large enough for us to draw any firm
conclusions, and we would welcome the opportunity to collaborate with users in extending our
knowledge base on any sex or ethnic differences in the use of Potentia.
© 2007 Knight Chapman Psychological Ltd. All rights reserved.
This Booklet may not be reproduced by any means without the written permission of the publisher.
KCP Potentia Series