international comparisons in senior secondary assessments
DESCRIPTION
Presentation at 37th IAEA Annual conference, ManilaTRANSCRIPT
![Page 1: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/1.jpg)
International comparisons in senior secondary assessments
Dennis OppossPresented at the 37th IAEA Annual Conference, Manila27 October 2011
![Page 2: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/2.jpg)
What is Ofqual?
We maintain standards, improve confidence and distribute information about qualifications, examinations and assessments used in England.
We recognise organisations that deliver qualifications and monitor their activities.
![Page 3: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/3.jpg)
Ofqual’s reliability programme
Two-year research study
![Page 4: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/4.jpg)
International comparisons work
Six-year research programme
Investigate the demand of assessments taken internationally in comparison to those taken in England
![Page 5: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/5.jpg)
International comparisons work
The first of these studies focuses on assessments available to senior secondary learners intending to progress to university in England
![Page 6: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/6.jpg)
The two types of studies
PISA, TIMSS, PIRLS
A special test designed to be a benchmark against itself
Test written through international collaboration to assess a common understanding of ‘quality’
Sample of all students
What Ofqual is looking at
What students do as part of their normal school assessment
Assessments written in the context of each system, reflecting their values, curriculum and purpose
Assessments for students looking to university
![Page 7: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/7.jpg)
Existing studies
‘On their own, cross-sectional international comparisons such as PISA cannot identify cause-and-effect relationships between certain factors and educational outcomes, especially in relation to the classroom and the processes of teaching and learning that take place there’.
OECD 2011
![Page 8: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/8.jpg)
Senior secondary assessment
Chemistry
English (national language or main language of tuition)
History
Mathematics
![Page 9: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/9.jpg)
![Page 10: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/10.jpg)
Other systems
International Baccalaureate
The ACT, USA
Cambridge Pre-U
Cambridge International
A levels
![Page 11: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/11.jpg)
Senior secondary assessment
Some assessments have the specific purpose of selecting students for higher education (for example a matriculation examination)
Others are assessments of educational achievement that in addition are used to identify students for higher education (of which A levels are one)
![Page 12: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/12.jpg)
Senior secondary assessment
The demand of each assessment was judged in relation to its appropriateness in preparing learners for entry to honours level degree study in England
![Page 13: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/13.jpg)
Assessment model level analysis
Reviewers consider
what learners needed to know and be able to do to succeed in the assessment
the nature of the subject matter to be covered
accessibility to the range of learners taking the assessment.
![Page 14: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/14.jpg)
Assessment instrument level analysis
CRAS analysis is used as a basis for organising the thoughts of reviewers with regard to specific features at question, task or question paper level.
![Page 15: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/15.jpg)
CRAS analysisCategory What is it? Scoring
1 2–3 4
C Complexity
The complexity of each component operation or idea and the links between them
Simple operations (i.e. ideas/steps)
No comprehension, except that required for natural language
No links between operations
Synthesis or evaluation of operations
Requires technical comprehension
Makes links between operations
R Resources
The use of data and information All and only the data/ information needed is given
Learner must generate all the necessary data/information
A Abstractness
The extent to which the learner deals with ideas rather than concrete objects or phenomena
Deals with concrete objects
Avoids need for technical terms
Highly abstract
Requires use of technical terms
S Strategy
The extent to which the learner devises (or selects) and maintains a strategy for tackling and answering the question
Strategy for answer is given
No need to monitor strategy
No selection of information required
No organisation required
Learner needs to devise their own strategy
Learner must monitor the application of their strategy
Learner must select content from a large, complex pool of information
Learner must organise how to communicate response
![Page 16: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/16.jpg)
CRAS analysis
Category: Resources
What is it?
The use of data and information.
Scoring:1 All and only the data/ information needed is given.
4 Learner must generate all the necessary data/information.
![Page 17: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/17.jpg)
CRAS analysis
Category: Abstractness
What is it?
The extent to which the learner deals with ideas rather than concrete objects or phenomena.
Scoring:1 Deals with concrete objects.
Avoids need for technical terms.
4 Highly abstract.Requires use of technical terms.
![Page 18: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/18.jpg)
Emerging findings
Independent research and extended essays
Internally set assessment
Multiple-choice questions
Mark schemes
Predictability
![Page 19: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/19.jpg)
Emerging findings – Mathematics
Pure mathematics or applications
Levels
New technology
Mechanics
![Page 20: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/20.jpg)
Emerging findings – Chemistry
Balance of coverage
Challenging topics
Mathematical content
![Page 21: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/21.jpg)
Emerging findings – English What is English?
What is a text?
Types of assessments
![Page 22: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/22.jpg)
Emerging findings – History What is history?
citizenship? study skills and concepts? rote learning facts and figures?
![Page 23: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/23.jpg)
Emerging findings – Summary
Layers of breadth and depth
Assessment variety
Producing the best A levels
![Page 24: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/24.jpg)
Future studies – 16+,11+
Criteria for selection of systems
Subjects: mathematics and English; others?
![Page 25: International comparisons in senior secondary assessments](https://reader033.vdocuments.us/reader033/viewer/2022061112/54574166af7959755d8b4f23/html5/thumbnails/25.jpg)
Getting in touch
www.ofqual.gov.uk