Download - The Importance of being Valid
![Page 1: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/1.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 1
The Importance of being Valid
Alastair Pollitt & Ayesha Ahmed
Cambridge Exam [email protected]
![Page 2: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/2.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 2
Defining ‘Validity’
There is no true meaning of’ Validity’
Definitions of ‘Validity’ are opinions about what the word should mean
How do we decide what ‘Validity’ should mean?
We need an inclusive concept of ‘Validity’ that is useful to everyone involved in the process of assessment, and not too far from what they normally mean by ‘Valid’.
A useful definition of ‘Validity’
![Page 3: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/3.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 3
The real world meanings of “valid”
1 offer 1022 ticket 323 licence 254 certificate 255 passport 196 law 157 voucher 14
. . . . . . . . . . . . . . .
18 reason 6
Relative Rank Concordance frequency
Source: Collins Cobuild Corpora (British + American)
![Page 4: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/4.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 4
What we mean by Validity
We have worked for many years with professionals who:
• Write questions; Review questions; Write mark schemes; Mark tests; Design tests; etc
The key issue for them, and us, is what the students’ minds are/were doing during the test –
in other words,
the Evidence that gives us about achievement.
![Page 5: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/5.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 5
What we mean by Validity
This is the essence of Validity – the interaction between a test question and the student’s mind.
If this goes wrong then Validity is impossible.
But how do we get it right?
By considering further the concept of Evidence.
![Page 6: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/6.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 6
Evidence
It is not enough to be Logically Valid.
A “widely used definition of assessment”:
The process of seeking and interpreting evidence for use by …
(Assessment Reform Group, 2002:
quoted by Stobart, 2009)
![Page 7: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/7.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 7
Importance Statement
A negotiated consensual statement of what is really important in the study of that subject during that phase of education.
Available to teachers, students, parents, examiners, employers, selectors – to everyone with an interest in the subject.
![Page 8: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/8.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 8
The Importance of Design & Technology
Design and technology prepares pupils to participate in tomorrow's rapidly changing technologies. They learn to think and intervene creatively to improve quality of life. The subject calls for pupils to become autonomous and creative problem solvers, as individuals and members of a team. They must look for needs, wants and opportunities and respond to them by developing a range of ideas and making products and systems.
They combine practical skills with an understanding of aesthetics, social and environmental issues, function and industrial practices. As they do so, they reflect on and evaluate present and past design and technology, its uses and effects. Through design and technology, all pupils can become discriminating and informed users of products, and become innovators.
![Page 9: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/9.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 9
The Importance of the Importance Statement
1 To ensure that teachers emphasise what is important
2 To ensure that test designers, question and mark scheme writers test what is important
3 To ensure that markers give credit for what is important
4 To ensure that test results are interpreted in terms of what is important
5 The source of Validity
![Page 10: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/10.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 10
A model of Validity
Validity is about measuring and interpreting what is important in education.
It is not merely about measuring and interpreting what is measured.
Our model describes where Validity comes from, and how it is maximised.
![Page 11: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/11.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 11
The Bucket Brigade model
Suppose: Y is a house on fire.
Suppose: X is a well full of water.
How do you put out the fire?
![Page 12: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/12.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 12
![Page 13: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/13.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 13
A typical assessment professional
![Page 14: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/14.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 14
The Entropy law of validity
Version 1
During the assessment process, validity always decreases
Version 2
Validity can only be lost, it can never be recovered.
![Page 15: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/15.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 15
How to maintain validity - 1
Writing the Importance Statement
![Page 16: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/16.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 16
How to maintain validity - 2
Specifying the test
![Page 17: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/17.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 17
How to maintain validity - 3
Writing the questions and mark schemes
![Page 18: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/18.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 18
How to maintain validity - 4
The students
![Page 19: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/19.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 19
How to maintain validity - 5
Marking
![Page 20: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/20.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 20
How to maintain validity - 6
Grading, etc
![Page 21: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/21.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 21
How to maintain validity - 7
Interpreting results
![Page 22: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/22.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 22
How to maintain validity
1 Writing the Importance Statement
2 Specifying the test
3 Writing the questions and mark schemes
4 Communicating with students
5 Marking
6 Grading
7 Interpreting
![Page 23: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/23.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 23
Summary
The meaning of test results depends crucially on a proper development of the test, so that it elicits evidence of what is important.
Without meaning, Validity is an empty concept.
The source of Validity must be a clear and shared concept of Importance.
![Page 24: The Importance of being Valid](https://reader036.vdocuments.us/reader036/viewer/2022070404/56813c64550346895da5f07a/html5/thumbnails/24.jpg)
Cambridge Exam Research Malta, 2009
© Pollitt & Ahmed, 2009 The Importance of being Valid 24
8The importance of being Valid
E
Q
i
M
Idea of a Question
Evaluation
Elicitation
IS
A
Importance Statement
TeachingT
Assessment
Evidence