plasc/npd user group mick quinlan ([email protected])[email protected] change, or the...
TRANSCRIPT
PLASC/NPD User Group
• Mick Quinlan ([email protected])
• Change, or the lack of it
• The Players: suppliers and users
• NC assessment process
• A wander through the ‘book’… an A-Z of the collection of NC assessment data
Forward
• I’d like to change the current process but:
– there are many players in the game each with existing systems and many dependencies
– we’re not good at managing change!
Drivers for Change
• 1998 Service delivery failure of data collection exercise
• 1999 Rose report – Weighing the Baby• 2004 Beasley report – service delivery failure of KS3
English test results• 2005 Statistics Commission report #23 – measuring
standards in English Primary schools• 2005 Statistics Commission report #26 – school
education statistics: user perspectives• 2006 NAO 3rd validation compendium report –
quality of data systems underpinning PSA targets– Broadly appropriate but system needs strengthening
Metadata
• Statistics Commission #23, Feb 2005, §11– We feel that public presentation of the KS scores in
statistical releases should include a clear statement about the uses to which the data may be put, and the limitations of the data in respect of those uses
• Statistics Commission #26, June 2005, §D– Full and informative statistical commentary should be
provided alongside the figures to guide users
Making Good Progress
Changes for 2006
• KS2/3 Electronic collection of TA, including pupil-level non-core results for KS3, via S2S – rather than OMR forms– A decision has been taken by the Department’s Head of
Profession for Statistics not to provide this data until the quality and robustness can be assured (SFR34-2006)
• KS3 English: Richard III in, Henry V out
• KS3 ICT onscreen test 2nd pilot (≈170,000 pupils)
The Players: DfES
• DfES – Paymaster/customer (TRD)– School Census/NPD and FSP/KS1 data collection– EduBase– School Self-Evaluation/Improvement
– Autumn Package (benchmarks, progress, target setting)
– PAT/RAISEonline)
– School Accountability - Performance Tables– SERAP, FORVUS, FFT
– National Statistics– Statistical First Releases
– PSA Targets
The Players: NAA
• A division of QCA (the Regulator)• Managers of the process:
– Test development (NFER, Edexcel, UCLES, RM)– Pupil registration (DfES, schools)– Test Printing/Distribution (COI/Granby)– Test marking (Edexcel)– Teacher Assessments (schools, LAs)– Data capture (Pearson)– Non-standard processes (in-house)
MarkScripts
Borderlining
Marked scripts
Level-setting
Samplescripts
Scriptscrutiny
Cut scores
Recom-mend-ation
DCresults
database
Borderlined scripts+ borderlined
marksheets+ school report
(inc. marks/levels)
Request forReview
R1: Clericalcheck
R2: IndividualRe-marking
R3/GR: Group review
Script +Revised
mark and level
Dataentry
MarkerStandardisation
Samplescripts
Guidance
Schools
Re-markedscript
Mark and level
Mark and level
7.2
7.1
7.4
7.7
7.6
7.12
7.10
7.9
7.8
7.11
Script pool collection
7.3
May-Jun
Level-settingdata collection
7.5
May/Jun
May/Jun
Jun
May
Jun
Jun Jun-Aug
Jul/Aug
Jul/Sep
Jul/Aug
Jul/Aug
Selected scripts
Returnedscripts
Specialconsideration
7.13May/Jun
Malpractice
7.14May/Jul
TeacherAssessments
7.15Jul
Request forProcess review
7.13Oct
Investigatereview
7.14Oct/Nov
Upheldreview
Re-review
The Process
Pretest 2:Draft Level thresholds
A is for Assessment
• National Curriculum– key stages: Foundation, KS1, KS2, KS3– Subjects
• core: English, maths & science• Attainment Targets• non-core: Art & Design, D&T, Geography, History,
ICT, MFL, Music and PE
– Test/Task & Teacher Assessment– Levels
Levels
• Q: What’s the difference between Level 4 and Level 5?
• A: 1 mark
• Making Good Progress (KS2-KS3,page 9)– This shows that 15[%] pupils did not progress
by a full level since finishing KS2 and three[%] have actually regressed
A
• A = pupil Absent from the test
• Aegrotat
• Achievement & Attainment Tables
• Age Standardised Scores (ASS)
• Additional time
• Appeals (process reviews)
BCD
• B = Working Below the level of the test• Borderlining• Braille• Compensatory award• Classes/Teaching Groups – collected in
1997• D = pupil Disapplied from the curriculum• Date of Birth
E
• E = Exceptional Performance
• Extension Papers
• Early takers
• Errors
FGH
• F = Future sitting (subjects early takers not taking)
• Fine grade
• Foot and mouth
• Gender
• Handwriting
IJKL
• Independent schools
• L = pupil Left school (KS2/3)
• L = below Level 2 in the reading comprehension test (KS1)
MNO
• M = Missing scripts• Maladministration/malpractice• Missing pages• Mental Maths test• Modified large print (MLP)• N = No test level (too few marks gained)• Names, of pupils• Overseas/Offshore schools• Optional tests (Y3-5 & Y7/8)
PQR
• P = Past sitting (subjects early takers took last year)
• Progress tests (Year 7)• P Scales• Q = result annulled - malpractice (DfES)• Reading• Reviews• Re-sits
ST
• Service Children’s Education (SCE) Schools
• Special Consideration• Shakespeare• Sub-Levels• Spelling test• T = Pupil unable to access the tests • Tier
UVW
• U = No TA subject Level calculated (KS1)
• Unique Pupil Number (UPN)
• V = mixed tiers
• Versions
• W = Working towards Level 1
• Writing
XYZ
• X = Marks pending (KS2/3)
• X = pupil not entered for reading comprehension test (KS1)
• Year groups
• Z = ineligible pupil