analysing ana results nicky roberts wits school of education kelello consulting...
TRANSCRIPT
![Page 2: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/2.jpg)
Context
Focus on Primary Maths project Errors in marking, adding and recording
results (2012) Using ANAs as a diagnostic tool for teaching
and development (2013)
![Page 3: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/3.jpg)
Research in two schools in Cape Town• Focus on Primary Maths 2012-2014
– School based maths teams– Professional development for FP teachers in
maths• Mini seminars• Co teaching interventions• Mathematical thinking course
– Responding to ANA attainment– Sharing lessons
![Page 4: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/4.jpg)
Responding to ANA attainment
Tracking ANA results year on year Question-by-question capture and analysis Planning for fourth term based on ANA
analysis Rasch analysis comparing the two schools
![Page 5: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/5.jpg)
Background to the two schools• 2 schools are equal partners in development project, chose
to be named at school level• Capricorn Primary School (CPS)
– 700 learners, 2011 below provincial mean (ANA and systemics)– Township, backyard dwellers, poverty, gangsterism and taxi
violence, drugs (Tic), alcohol (FAS). Mostly households in poverty.
– Minority not English main lang, LOLT is English• The Grove Primary school
– 700 learners, history of excellent attainment (ANA and systemics)
– Suburban, model C, mostly affluent households– Majority English main lang, LOLT is English
![Page 6: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/6.jpg)
No anchor questions Predictable test Not year-on-year
comparison Assume system is
stable, then performance relative to national mean is indicator of overall performance
![Page 7: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/7.jpg)
Indicator of proportion of learners meeting grade level requirements (relative to national pass rate, on that ANA)
![Page 8: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/8.jpg)
WCED systemic (standardized questions and marking, so year-on-year comparison)
![Page 9: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/9.jpg)
Question by question data capture
Real power for instructional design
Q1 Q2 Q3 Q4...
Learner 1Learner 2Learner 3Learner 4Learner 5….
CorrectIncorrectFacility
![Page 10: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/10.jpg)
ANA analysis template
Class capture sheet Summary sheet Question summary sheet Spreadsheet template suburban school Spreadsheet template township school
![Page 11: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/11.jpg)
What we did in 2013
Before ANA marking Import the learner names from EMIS into classes. One class
per sheet. Check formulas and that the template works as you want it
to
On the marking day Laptops/ computers with template in marking venue Capture marks for each script Queries – go back to teacher who was marking
![Page 12: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/12.jpg)
What we did in 2013
After marking daysCreate grade summariesCompare to previous yearsLook for questions of common difficulty
Very poor: colour code in green Poor: colour code code in pink Revise maths teaching and development plan for
fourth term
![Page 13: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/13.jpg)
Functions used in Excel template
SUM AVERAGE MIN MAX STDEV
Some notes on using Excel
IF-THEN COUNT IF
![Page 14: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/14.jpg)
What we did in 2014
Repeated the descriptive statisticsUsed this to create grade level ‘mini lesson studies’
cycles, eg: Teaching bonds in Grade 1 Teaching views and positional language in Grade 2 Teaching analogue time in Grade 3
Plan – teach- reflect cycleTeacher presentations at mini conference, AMESA
regional and AMESA national
![Page 15: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/15.jpg)
![Page 16: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/16.jpg)
Mini conference (Day 1 internal)
![Page 17: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/17.jpg)
Mini conference (Day 2 district)
![Page 18: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/18.jpg)
![Page 19: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/19.jpg)
Mini conference
![Page 20: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/20.jpg)
AMESA regional
![Page 21: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/21.jpg)
What we did in 2015
Repeated the descriptive statistics Used the data to conduct Rasch analysis:
Misfitting itemsBiases between schools (based on DIF values)
Used this to target problematic questions and review these qualitatively
![Page 22: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/22.jpg)
Qualitative analysis (from initial qunat data)
Total marks 1Mean 0.02Standard deviation 0.14Mean percentage 2%Number writing 80Number getting 0 79Percentage getting 0 99%
Identified as a major area of weakness
![Page 23: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/23.jpg)
Errors on Q 25.1
Some completely off
Most common error was to break up number rather than converting
![Page 24: Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting nickyroberts@icon.co.za](https://reader035.vdocuments.us/reader035/viewer/2022081520/5697bfb91a28abf838c9fce6/html5/thumbnails/24.jpg)
Conclusion
ANA is a major investment in SA education It gives annual standardised data across schools If anchor questions were included, it would also give year-on
year comparisons It gives schools rich data for error analysis (if item level analysis
is conducted) which can lead to lesson study cycles/ interventions
Teachers need support in marking ANAs (again directs attention to the mathematics) and reporting common errors to district/ province