Assessing the Impact of the Ishraq Program in Egypt

Ghada Barsoum, PhDSenior Program

Manager Poverty, Gender and

Youth ProgramPopulation Council

International Initiative for Impact Evaluation

ConferenceApril 2009

Ishraq: Basic Facts Targets out-of-school girls (12-15) In rural Upper Egypt: Lowest human

development indicators: 26% of girls receive no schooling (ELMPS06); unpaid family enterprise workers; early marriage and childbearing; FGM

Intervention lasts for 20 months Girls meet 4 times weekly for 3 hours

Ishraq: Basic Facts First piloted in 2001 by four Int’l NGOs:

Save the Children; CEDPA (the Center for Development and Population Activities); Caritas; Population Council

Government partner with an increasing role: National Council of Youth (NCY) to allocate specific hours in youth centers to be used as “safe spaces” for girls

Local NGOs implementing the model and mobilizing communities on the village level

Ishraq: Basic FactsProgram Components:

1) Literacy (Learn to be Free, Caritas): Girls sit for the literacy exam and are given the opportunity to be mainstreamed into the formal educational system (grade 7) – Program provides help in issuing birth certificates (required for school enrollment)

2) Life-Skills (RH, Basic: livelihood, nutrition, girls rights .. (CEDPA)

3) Sports and Recreation (PC)4) Financial Literacy (PC) (new)5) Free Meals and food rations (new)

Intervention so far .. Piloted in 2001 in 4 villages in Minia

(N=275); expanded in 2004 to 6 villages in Minia (2nd rounds in 2 villages from the pilot stage) (N=336); expanded in 2006 to 5 villages in Beni Suef (N=289)

Currently in its scale-up and –out phase in 50 villages (2500 girls) and in full partnership with NCY with an institutionalization plan for sustainability

Rigorous Impact Evaluation

has been Central to Ishraq

in both its Pilot and Scale-up Phases

Impact Evaluation in the Earlier Phase

Baseline and endline surveys in control and intervention villagesMost of the analysis was descriptive due to

limited sample size. Positive outcomes in:Functional literacy, cognitive skills and

continuing schooling; 92% of those who sat for the literacy exam passed; over half

achieved “Excellent” score; 66% (re-)entered the formal education system (compared to a national average of 6% for those who pass literacy exams)

Positive gender norm attitudes; knowledge about nutrition, hygiene, girl’s rights & RH; against FGM

Impact Evaluation Plan for the Scale-up PhasePlan and Challenges

Combined Methodology1. Baseline and endline surveys in

control and intervention villages2. Combining Monitoring data (such

as participation and attendance levels) with impact evaluation data

3. Qualitative research

Self-Selection Bias Concerns Girls who self-select into the program are

more likely to be better off. Selection bias problem results in mixing-up the treatment effect with pre-program difference in attributes.

To estimate the impact of the program we need to know the difference between:(1) the outcomes for participants if they

participated in the program and (2) the outcomes had they not participated

Pre-analysis Activities Addressing Selection Bias1. Girls’ self-selection into

participation and recruitment activities

2. Village selection3. Questionnaire design 4. Data collection plan

1. Recruitment Activities and Selection Bias1. Recruitment: debate on whether first-

come-first-served or randomization based on hh listing

2. Rigorous community mobilization to eliminate the impact of:

1. Geographical proximity2. Friends and relatives

2. Improved Ishraq recruitment strategy through visual advertisements; trained promoters; village committees; village-level workshops

2. Village Selection ProcessRandomization in the selection of

intervention villages from a list of eligible villages. Eligibility criteria: size of out-of-school cohort, presence of a youth center and a school

Statistical power to determine the number of control villages

Selection of control villages from the very same list of eligible villages

3. Questionnaire DesignAdding instrumental variables to household and individual characteristics and attitudesInstruments that affects participation but not directly affect outcomes. Distance between home and youth center (proxy for costs of participation).

+ Distance to the village center as an explanatory variable in both the participation and outcome equations to capture access to other services and the effect of the centrality of the household location.

4. Data Collection Plan at Baseline and EndlineA saturation sample of out-of-school

girls in intervention and control villages based on listing and mapping of all hhs in village

Design allows us to compare the characteristics of control village and intervention village members; and the characteristics of participants and non-participants in intervention villages

Data from M&E tools to be incorporated into endline dataset

Examined Outcomes

Literacylevel of success in passing government-

sponsored literacy tests and rates at which girls are mainstreamed back into preparatory (middle) schools.

Attitudes about marriage and childbearing:Ideal age at marriage Views about decision-making regarding

marriage partners and timing of marriage. Ideal family size and fertility intentions.

Examined Outcomes- CONTDKnowledge about nutrition, hygiene, and

reproductive health.Attitudes about harmful traditional practices

(e.g., FGM)Social isolation, peer networks, and

participation in group or community activitiesGender norms index

Qualitative Data CollectionConfirming and enriching findingsMerging the two methodologies in the

analysis and write up of resultsOpen-ended semi-structured interviews with

girls joining the project, drop-out cases and their families

Focus group discussionsInforms program design and constitutes an

ongoing process

Thank You