combining test grades based on cut scores with qualitative points oslo, september 2015 samuel...
TRANSCRIPT
Combining test grades based on cut scores
with qualitative pointsOslo, September 2015
Samuel Sollerman
The Swedish national tests in mathematics
Some notes about the Swedish national test system:
- The teacher is responsible for all assessment and grading of the student.
- We have no formal examinations, there are national tests to support the teacher.
- No central marking of national test, the teacher marks and assess the national test.
The aims of the national tests (from the Swedish agency of education):
- Support a equal and fair assessment and grading.
- Provide a basis for an analysis of the extent to which the knowledge requirements are met at school level, on the principal organiser level and at national level.
- The national tests can also help to concretise the curriculum and to increased goal fulfilment for the students.
The Swedish Grading Scale
AB
CD
EF
E
C
A
Sweden has a goal and criterion referenced grading system.
The grade should express to what extent the pupil has met the knowledge, requirements stated for each subject and course.
There are national knowledge requirements
for grades E, C and A.
Qualitative points
- There are national knowledge requirements explicitly described for grades E, C and A.
- In national test in mathematics there is qualitative points that corresponds to each of these levels.
- The points is called E-points, C-points and A-points.
Why qualitative points?
Concretises the knowledge requirements for both teachers and students and illustrates the progression in the knowledge requirements.
Example:The number of visitors at a web site increases by the same
(1/1/1)percentage each year, two years in a row. Find the yearly percentage increase if the total percentage increase for the two-year period is 37 %.
Cut scores with qualitative points
The cut scores is constructed based on the amount of points of the whole test, with demands of
- A requirement of a certain amount of total points
- A requirement of a certain amount of points on an certain level
Example:Maximum 31 E-, 39 C- and 23 A-points, totally 93 points.Lower limit for the test gradeE: at least 20 points at level E or higher.D: at least 35 points of which at least 13 points at level C or higher.C: at least 47 points of which at least 22 points at level C or higher.B: at least 59 points of which at least 7 points at level A.A: at least 72 points of which at least 13 points at level A.
Why cut scores with qualitative points?
Makes it possible to require both width and depth of showed knowledge.
Cut scores with qualitative points
The standard setting processThe standard setting is done before the test is used.
Standard setting with panels with experts (teachers).
Total requirements e.g. Modified Angoff and The Direct Consensus Method.
Level requirements e.g. Modified Angoff with Yes/No and The Direct Consensus Method.
Completed with analyses of data from pretesting and linked items.
Cut scores with qualitative points
An example of the outcome
Results from a national test in mathematics,
Upper secondary school, course “Mathematics 1b”
Random sample of students, n=3831
Test grades compared with preliminary grades
Correlation: 0,895Same grade: 68 %Higher grade: 19 %Lower grade: 12 %
n = 3831
n = 3831
Test grades compared with preliminary grades
Correlation: 0,929Same grade: 73 %Higher grade: 21 %
Lower grade: 5 %
Students with higher total but not higher level
n = 3831
Test grades compared
with preliminary grades
Correlation: 0,879Same grade: 71 %Higher grade: 24 %
Lower grade: 4 %
Students with higher level but not higher total
The cut scores (has moved from interval to strict cut scores).
Is it more equal and fair?
What is the signal to the student and to the teacher?
The cut scores based on qualitative points.
Do they help to decide test grades more accurate?
Do they help to support the teachers grading and to concretise the curriculum?
The cut scores on Swedish national tests in mathematics
How to design tests and cut scores so that they become a real support for the teachers grading, yet are equal and fair?
Contact:Samuel [email protected]