Multi-Country Evaluation for UNICEF: Increasing access and equity in early

childhood education in six CEE/CIS countries

Education for Change September 2015

Introduction and Context

Presenting evaluation for UNICEF CEE/CIS ROMethodResults Follow-up

ContextUNICEF CEE/CIS shift from service delivery to system

changesRegional generic TOC for such change5 multi-country evaluations commissioned by CEE/CIS

Evaluation on ELSR Objectives:

Are changes in Early Learning and School Readiness (ELSR) systems leading to increased access and reduced equity gaps?

Contribution of UNICEF to system-level changes Validity of UNICEF Theory of Change (TOC)

6 cases countriesArmenia, Bosnia and Herzegovina, the former Yugoslav

Republic of Macedonia, Kosovo (UNSCR 1244), Kyrgyzstan, Moldova

Evaluation Methodology Developed country and regional TOC for ELSR Review and analysis of national data sets Document review

Country case visits:National level interviews Sub-national authoritiesELSR institutions, teachers and parents

Theory of Change

Impact(Impact at national level – not direct UNICEF contributions)

Huge data constraints on understanding marginalised Overall access increased, but often starting from a low base

Moldova: NER in preschool (3-6 years old)

Armenia: NER in preschool (3-5 years old)

Kyrgyzstan: NER in preschool (3-6 years old)

2005

2006

2007

2008

2009

2010

2011

2012

0%10%20%30%40%50%60%70%80%20

07

2008

2009

2010

2011

2012

0%10%20%30%40%50%60%70%80%

2005

2006

2007

2008

2009

2010

2011

0%10%20%30%40%50%60%70%80%

Impact continued… Sub-national equity gaps in access vary country by country Urban/rural disparities remain

Rural/urban rates of preschool enrolment (Ar, Mo, Kg) or attendance (FMa, BH)

2007 20120%

10%20%30%40%50%60%70%80%90%

100%

Armenia

2005 2012

Moldova

2005 2011

Kyrgyzstan

2006 2011

Bosnia and Herzegovina

2005 2011

The former Yugoslav Republic of Macedonia

Impact continued… Gender parity index for enrolment slightly in favour of girls

200520062007200820092010201120120.9

1

1.1

1.2Armenia

Kosovo UNSCR 1244

Kyrgyzstan

Moldova

The fomer Yugoslav Republic of Macedonia

GPI

Impact continued… Where we have data on ethnicities, still wide

disparities

2011 National and Roma preschool attendance rates for 3-4 year olds (MICS data)

Male

Female

Total

-5% 0% 5% 10% 15% 20% 25% 30%

Bosnia and Herzegovina

Male

Female

Total

0% 5% 10% 15% 20% 25% 30%

The former Yugoslav Republic of Macedonia

National

Roma

Impact continued… Appears equity gap between richest and poorest widening

Number of children with disabilities enrolled slowly increasing

Preschool attendance for 3-4 year olds by wealth quintiles (MICS data)

The former Yugoslav Republic of Macedonia Bosnia and Herzegovina

2006 20110%

20%

40%

60%

Second

2005 20110%

20%

40%

60%

Impact continued…Limited data on effects of preschool

“School-readiness” increasing but little dataWidespread Grade 0 years in primary facilities

System changes

Social norms: Importance of education recognisedExpectations of integrated servicesBottleneck around marginalised groups

Legislation and policy:Frameworks and ELDS in place

System changes continued….Budget:

Rising , leveraging external capital funding (GPE) Budgetary responsibility often decentralised

Management and coordination: Examples of strong political will and high level

coordination Cross-sectoral approach still embryonic Decentralised capacities weak Data and inspection systems very weak

Availability and access: Substantial increase in preschool

facilities and staffing (Grade 0 push)

Financial access: Despite official free provision, financial barriers

still considerable

Quality: Growing work with standards, training and some

on certification of teachers

System changes continued….

Voice for children: Strong advocate for preschool provisionLimited work on voice of children

Policy advice and TA:Strong role in legislation, policy, ELDS,

facilities, trainingCapacity support, but decentralised capacities

require more holistic approach

UNICEF contributions

Monitoring and EvaluationGrowing body of studies supplying evidenceMonitoring sector remains a challenge No data on outcomes

UNICEF contributions continued…

National dialogue:Facilitating national and local level discussionsPromoting cross-sector dialogue• Constrained by institutional divisions and own

organisation Knowledge exchange:

Strong reputation for knowledge brokeringCross-country and cross-municipal

learning been important but limited

UNICEF contributions continued…

UNICEF contributions continued… Modelling:

Provides the credibility and exemplarsLow-cost alternative provision and financing studiesModelling of teaching approaches and materials

Follow up: vision and strategy Vision and strategy recognising:

Clear equity framework Coherence of provision for whole 3-6 age range Articulation with broader 0-8 early childhood period Articulation between care and education for 0-6 years Cross-sectoral agenda linking ELSR with child

protection, education, social protection, health, M&E Engagement with mushrooming private sector HRBA for ELSR Quality at heart of increased access

Follow-upStrategy addresses Quality

quality risks from rapidly expanding access standards applicable/applied across types of provision work with ELSR professionals on children’s ways of learning

and respect for children’s rights guidance/tools for child participation for ages 4-8

Ability to work with decentralised systems: partnerships with line ministries for decentralisation and sub-

national authorities capacity development strategy for system institutions

Follow-up on data and research

Develop data protocols for the sector

Research agenda on:Understanding school readinessOutcomes of varying forms of provisionDifferentiation of marginalised groupsParental attitudes

THANK YOU!Evaluation team:

John Wood – EfC DirectorJake Grout-Smith – formerly EfC Consultant and Projects ManagerProf. Helen Penn – Prof. Emeritus, UELDr. Anise Waljee – Independent consultantLaetitia Antonowicz – EfC Associate, consultantSophie Tanner – formerly EfC Research and Project Manager

2014 CEE/CIS and Baltic States: Multi-Country Evaluation (MCE): Increased Access and Equity in Early Childhood Education

http://www.unicef.org/evaluation/index_BestEvals.html

http://www.unicef.org/evaldatabase/index_81158.html

EfC: www.efc.co.uk