The Evaluation of Programs Combating Female Genital Cutting:

Data Sources and Options

February 6, 2007

P. Stanley YoderMacro

International

Overview of presentation• Assessing long-term impact

• DHS data

• Single survey data

• Challenges in designing evaluations

Current situation: Anti-FGC programs

• Great variety of program types- Public awareness media campaigns- Social mobilization in specific populations- Lobbying for legal action- Enlisting help from traditional and political leaders

• Very little empirical evidence of impact (Diop & Askew, book chapter in R.M. Abusharaf,

ed. 2006)

Need for data on FGC program impact

• Statistical evidence for changes in FGC prevalence

• Possible use of DHS data?

• Surveys of sample populations- Baseline and follow-up surveys- Comparison of intervention and control regions

Challenges in collection of relevant data

• Conducting a survey in program areas only

• Need to have data on events 10 years ago

• Temptation to rely on program personnel rather than outside observers to evaluate

• DHS provides data on women 15-49 years old and not on young girls

• Need to describe situation taking account of secular trend

Use of DHS data for FGC evaluations

• Data on prevalence by age group of women

• Data on daughters (most recently cut)- Age at FGC- Identity of practitioner- type of cutting

• Does not provide data on all daughters

DHS data on prevalence by age group

Senegal Kenya Nigeria Tanzania

15-19 24.8% 20.3% 12.9% 9.1%20-24 28.0% 24.8% 17.0% 13.7%25-29 28.4% 33.0% 20.8% 15.2%30-34 30.1% 38.1% 19.4% 16.0%35-39 30.5% 39.7% 22.2% 16.0%40-44 30.3% 47.5% 22.2% 18.8%45-49 30.6% 47.7% 28.4% 22.9%Total 28.2% 32.2% 19.0% 14.6%

DHS data on distribution of FGC

• Prevalence by region

• Prevalence by ethnicity

• Prevalence by rural/urban residence

• Prevalence by religion (sometimes)

Example of DHS data for Senegal: Ethnicity

FGC prevalence

Percent of total pop.

Wolof 1,6% 40%Serer 1,8% 16%Diola 60% 5%Fulani 62% 25%Mandinka 74% 5%Soninke 78% 3%

Limitations in use of DHS data for evaluations

• No complete data on FGC among young girls

• Sample size limits aggregation by small units such as those covered by an FGC program

• Provides data on FGC events 10-15 years ago and more only (women 15-24 years old)

Assessing long-term impact: Stand alone surveys

• Baseline and follow-up surveys- relatively expensive to design and implement- results after more than five years

• One survey with intervention & control areas - much cheaper- difficult to identify intervention & control areas, for they don’t follow administrative units- very hard to match intervention & control areas

Example: Tostan evaluation for UNICEF New York

• Commissioned and funded by Child Protection Division of UNICEF New York

• Implemented through contract between UNICEF and Macro International

• Additional funding by USAID

• Two components: Quantitative and Qualitative- Quantitative: Centre de Recherche pour le Développement Humain (CRDH)- Qualitative: Population Council, Dakar

Quantitative component: intervention and control areas

• Designed and implemented by the CRDH

• Questionnaire for households and for women 15-49 years old

• Questionnaire similar to DHS

• Questions on knowing about Tostan, participation in Tostan, knowing about a Public Declaration, participation in a PD

Basic questions to answer

• How has Tostan affected:- average age of marriage?- cutting of young girls?

• How has participation in a Public Declaration affected:- average age of marriage?- cutting of young girls?

Three types of villages where FGC is practiced

Basic comparison:Data on age of first marriage and rates of FGC in three types of villages:

• Type A: Tostan operated and population participated in a Public Declaration (PD)

• Type B: Population participated in PD without Tostan

• Type C: Control villages: No Tostan, no PD

Assessment of program impact

• What are the starting points for rates of FGC and for age of marriage in these three types of villages?

• What is the secular trend in these areas?

• How do the intervention and the control villages differ?

• To what extent can differences be attributed to Tostan activities?

Village type by prevalence of FGC for women

• Type A villages – FGC prevalence 64%

• Type B villages – FGC prevalence 81%

• Type C villages – FGC prevalence 87%

• Implication: The point of departure is different for the assessment of any change that may have occurred

Major challenges for village comparisons

• Matching control and intervention villages by:- ethnic composition- practice of FGC- literacy- development of social services

• Locating villages near Tostan villages that had no Tostan influence

Reporting on an evaluation: Being clear

• The questions to be answered

• The indicators to be followed

• How and why they are important

• The expectations of the program personnel

• The ways evaluators would assess the program

• The ways evaluators would measure success

Setting up a new evaluation

• Questionnaire that matches program activities

• Sample population that covers program areas plus control areas

• Choosing indicators that show process as well as impact

• Identifying a neutral agency to conduct survey

• Articulating how assessments will be made

• Plan final report that provides all this information