School Quality Assurance: Developing a scalable model for school self-improvement in Madhya Pradesh

September 2015

Introduction

• Ark is an international organization whose purpose is to transform children’s lives

• We have worked with the Government of Madhya Pradesh since 2012 to develop a School Quality Assurance Framework

• The Framework is currently being rolled out to 20,000 schools, and should reach the entire state by 2018

Identified Problems and Proposed Solution

The Problem• The quality of education in

Madhya Pradesh is extremely low – according to ASER, in the bottom 5 states in India

• Our hypothesis was that this is driven by three factors:1. A focus on provisions delivery

rather than quality of teaching

2. A lack of effective governance and robust accountability structures

3. Very limited capacity in the system to support school improvement

Proposed Solutions• Define a set of school

quality standards centred on teaching

• Develop a simple assessment tool to focus policymakers on accountability & transparency

• Embed the assessment tool in a framework that: – Gives local administrators

practical tools to support schools

– Helps school leaders and teachers understand their challenges and self-improve

Quality Assurance Framework: Approach

Quality Assurance Framework:Content

The Standards

• 2 Assessors: one external expert, one local administrator

• One-day assessment process; nine tools used with leaders, teachers, children and parents

• Automated report generation – does not depend on assessor judgement

The Process

Designing for Scale

• Designing for scale– Focus on the essentials of the problem and

available resources– Stake holder buy-in at all levels is critical

• Rapid Prototyping– Fail-Fast mentality

• Accelerating– Government ownership from design phase– Building credibility through constant self-

assessment and evaluation

Initial Findings 1:Pitched at the right level

Below Close Meet Exceed0%

10%

20%

30%

40%

50%

60%

70%

80%

Overall School Ratings

OverallPrimarySecondary

• The SQA assessment tool focuses attention on the need for all schools to improve, but without causing a political backlash

Initial Findings 2:Focuses on the right issues

Leadership Teaching & Learning Outcomes: Academic Overall0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage Schools performing Be-low on 4 key Indicators

Axis Title

• The tool focuses attention on teaching & learning as the single critical area for improvement

Initial Findings 3:Enables policy insight

• Assessment is low-stakes but transparency can nevertheless improve administration

Initial Findings 4:Potential to shape expectations

• The system has limited understanding of what “good teaching” looks like; head teachers are unable to offer leadership of teaching

• The process of using the tool has the potential to build an understanding of “good teaching” among teachers, leaders and administrators

Formal Evaluation

• Strong technical leadership from Professor Karthik Muralidharan and Dr. Abhijeet Singh

• Randomized at the “cluster” level to eliminate spill over effects

• Multi-dimensional outcome variables:– Process: Student, Teacher and School Level– Student Level Academic Outcomes– Management Processes

4 Key Lessons

• Stay flexible: a collaborative approach with rapid prototyping and a “fail fast & smart” mentality

• Design for scale: keep costs low, tailor to human resource constraints & build stakeholder support

• Focus on a few key issues: political impact can be as important as technical accuracy

• Have a clear vision: low-stakes accountability but building capacity to support schools and change mind-sets

Next Steps

• Madhya Pradesh: – Reaching out to all 120,000 schools by 2018– Investing in improved training for local

administrators

• International:– Version 2.0 in development, incorporating

improvements for increased rigour & challenge– Expansion to new geographies, with local

adaptations– Developing a new Early Years tool and framework