Download - Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India
![Page 1: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/1.jpg)
Pitfalls of Participatory Programs:
Evidence from a randomized evaluation in education in
India
Abhijit Banerjee (MIT)Rukmini Banerji (Pratham)
Esther Duflo (MIT)Rachel Glennerster (MIT)
Stuti Khemani (The World Bank)
![Page 2: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/2.jpg)
Motivation
Community participation is being pursued in policies and programs to improve the quality of public services
We evaluate the impact of mobilizing communities (using information & advocacy) to participate in publicly provided education services
![Page 3: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/3.jpg)
The Study Location In Uttar Pradesh (UP)—a “typical” district, Jaunpur
![Page 4: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/4.jpg)
Institutional Context in UP Village-level education committees
(VECs), representing parents and village leaders, created by state education policy in India since the 1990s
VECs have specific powers to monitor quality, and manage public resources to improve quality
Can recruit community-based additional teachers
![Page 5: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/5.jpg)
Findings from Baseline Survey Baseline Survey (March – June 2005), 280
villages:• Learning outcomes
30 households randomly selected in each village
all children between the ages of 7 and 14 tested on basic reading, writing, and math skills
• Community participation and local governance All VEC members interviewed
All government school head teachers interviewed
10 households (from among the above 30) interviewed
• School resources and school functioning All government schools surveyed through interviews and
direct observation
![Page 6: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/6.jpg)
Parents don’t know that a VEC exists
Have you heard of the VEC?
1.5%1.1%
5.0%
7.6%92.4%
Villagers Who Don't Know of aVillage Education Committee
Villagers Who Think there is aVEC
Villagers Who Believe there is aVEC, But Can't Name Any VECMembers
Villagers Who Can Name OnlyOne or Two VEC Members (the Pradhan and/orHeadmaster)Villagers Who Can Name MoreVEC Members than Just thePradhan and Headmaster
* Based on 2,803 household surveys in 4 random blocks in the District of Jaunpur, UP. Each household is weighted by total number of households in village divided by number households surveyed in village.
![Page 7: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/7.jpg)
VEC members don’t know their roles
Percent of VEC Members Who:
Don’t know that they are members of the
VEC
Don’t know that funds are
provided to VECs to
improve schools
Don’t list hiring of community-
based additional teachers as a
VEC instrument
Head Teachers 4.2% 4.2% 86%
Other VEC Members 22.7% 73.4% 95%
![Page 8: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/8.jpg)
Parents of children at low levels of learning tend not to know this…
Perception versus Reality: Read
34%
6%
30%
22%
5%
10%
17%
11%
4%
10%
19%
23%
17%
8%
16%
36%
61%
78%
91%
1%1%
0%
1%0%1%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 1 2 3 4
Actual Read Level
Pa
ren
ts' P
erc
ep
tio
n o
f th
eir
Ch
ild
ren
's R
ea
d L
eve
l
Perceived Read Level = 4
Perceived Read Level = 3
Perceived Read Level = 2
Perceived Read Level = 1
Perceived Read Level = 0
* Data based on surveys of 2,803 households, and the testing of 5,377 children in 4 random blocks in the District of Jaunpur, UP. Child tests are weighted by number of children in village divided by number of children tested in surveyed households.
![Page 9: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/9.jpg)
Interventions to Encourage Participation: (1)
Mobilizing communities through advocacy, and information about VECs
Small, informal meetings in each hamlet during 2 days in a village
Village-wide meeting on 2nd or 3rd day, with participation of key VEC members—Village Head and School Teacher
Distribution of pamphlets to VEC members listing and explaining their roles
![Page 10: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/10.jpg)
Interventions to Encourage Participation: (2)
(1) + Testing:
Reading test for outcomes information, building monitoring capacity
In hamlet meetings, facilitators begin testing children; community invited to test children themselves and prepare hamlet-level “report cards”
In village-wide meeting, hamlet volunteers invited to present testing tools and “report cards”
![Page 11: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/11.jpg)
Interventions to Strengthen Community Participation: (3)
(2) + Teaching :
Capacity Building to Directly Improve Learning
In village meetings, facilitators present Pratham’s program for improving reading
Offer to train any volunteers to hold reading classes
![Page 12: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/12.jpg)
Experience with Interventions 65 villages each received Interventions 1,
2, and 3 between September and December 2005
Soon after a new village government was elected, and could constitute new VECs
Repeat visits in Jan-Feb 2006 to hand-out and explain pamphlets to VEC members
85 villages served as controls
![Page 13: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/13.jpg)
Experience with Teaching Intervention
Local youth volunteered: 405 reading classes held
No. of Reading Classes in Teaching Intervention Villages
10
1
2
4
6
3
10
7
3
4 4
3 3 3
2
0
2
4
6
8
10
12
0 1 2 3 4 5 6 7 8 9 10 11 12 13 16
No. of Reading Camps
No. of Villages
![Page 14: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/14.jpg)
Distribution of the Size of the Reading Camps
2 1 1 1 24
119
24
18
52
32
25
37
19
28 2724
1517
41
11
2 1 1
0
10
20
30
40
50
60
4 5 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Number of Students
Num
ber
of C
lass
es
Participation of 7453 children in reading camps in 55 villages: 135 children per village
![Page 15: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/15.jpg)
Impact EvaluationEnd-line survey took place over March-June 2006
Specification (for each of K “families” of outcomes):
X: baseline levels of every outcome in the family Difference-in-Difference estimates White standard errors clustered by village Additional controls in some specifications (parent’s caste,
education, occupation, literacy; child’s age, gender, school status)
321 321 TreatTreatTreaty kkkijk
ijkkX
![Page 16: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/16.jpg)
BaselineMean in
ComparisonMean Group Treatment 1 Treatment 2 Treatment 3 Any Treatment
(1) (2) (3) (4) (5) (6)Family A. Dependent Variables - VEC members information about their roleMentioned that they are 0.383 0.247 0.084 0.083 0.030 0.066in the VEC unprompted (0.024) (0.038) (0.060) (0.061) (0.058) (0.046)Mentioned that they are in 0.753 0.602 0.065 0.095 0.047 0.070the VEC when prompted (0.020) (0.044) (0.067) (0.061) (0.064) (0.051)Had heard of SSA 0.258 0.209 0.101 * 0.062 0.065 0.075 *
(0.018) (0.033) (0.056) (0.053) (0.058) (0.042)Knew that their school can 0.210 0.179 0.119 ** 0.048 0.072 0.078 *receive money from SSA (0.017) (0.033) (0.056) (0.049) (0.057) (0.041)Had received VEC training 0.132 0.046 0.118 *** 0.135 *** 0.148 *** 0.134 ***
(0.016) (0.020) (0.042) (0.044) (0.041) (0.030)Average over family of 1.868 *** 1.653 ** 1.468 ** 2.227 ***outcomes (in standard deviations) (0.695) (0.660) (0.714) (0.661)
Family B. Dependent Variables - VEC member activism Complained 0.131 0.099 -0.037 0.029 0.034 0.010
(0.015) (0.026) (0.036) (0.044) (0.043) (0.033)Raised money 0.050 0.025 -0.021 -0.030 ** -0.022 -0.024 *
(0.010) (0.012) (0.015) (0.014) (0.015) (0.014)Number of school 6.444 8.208 -1.182 -1.585 -1.740 -1.518inspections reported (0.565) (1.216) (1.766) (1.665) (1.824) (1.485)Distributed scholarships 0.063 0.034 -0.027 0.018 -0.026 -0.012
(0.011) (0.014) (0.036) (0.045) (0.037) (0.033)Implemented midday meal 0.111 0.104 0.009 0.013 0.026 0.016
(0.015) (0.026) (0.029) (0.024) (0.025) (0.019)Average over family of -0.722 -0.303 -0.261 -0.407outcomes (in standard deviations) (0.534) (0.542) (0.528) (0.538)
OLS : impact of treatment
Table 1: Impact on VEC Awarness and Activities
![Page 17: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/17.jpg)
BaselineMean in
ComparisonMean Group Treatment 1 Treatment 2 Treatment 3 Any Treatment
(1) (2) (3) (4) (5) (6)Family C. Dependent Variables - VEC member knowledge about the education situation in the villageDidn't know about the 0.089 0.064 -0.056 ** -0.018 -0.044 -0.039"paragraph" question (0.012) (0.021) (0.027) (0.035) (0.028) (0.026)Didn't know about the 0.094 0.061 -0.055 ** -0.009 -0.043 -0.036"sentence" question (0.013) (0.021) (0.027) (0.036) (0.028) (0.026)Perception minus reality of how many 0.153 0.098 -0.064 * -0.033 -0.060 * -0.051 *kids can read paragraphs (0.012) (0.024) (0.037) (0.034) (0.034) (0.030)Perception minus reality of how many 0.118 -0.001 -0.040 -0.002 -0.024 -0.021kids can write sentences (0.012) (0.022) (0.036) (0.032) (0.031) (0.028)Average over family of -1.760 ** -0.458 -1.423 ** -1.357 *outcomes (in standard deviations) (0.725) (0.744) (0.719) (0.743)
Family D: VEC member knowledge about their responsibilities & whether they have acted upon them: Shiksha MitraMentioned that hiring a shiksha 0.025 0.041 -0.015 -0.017 -0.026 -0.019mitra is a VEC responsibility (0.008) (0.016) (0.022) (0.020) (0.018) (0.017)Hired a shiksha mitra last yr 0.016 0.016 -0.002 0.007 -0.002 0.001
(0.005) (0.009) (0.012) (0.015) (0.013) (0.011)Claimed that the VEC will hire 0.009 0.016 0.004 -0.003 -0.018 * -0.006a shiksha mitra next yr (0.004) (0.009) (0.016) (0.012) (0.010) (0.010)Average over family of -0.198 -0.238 -1.161 * -0.542outcomes (in standard deviations) (0.584) (0.601) (0.674) (0.631)
OLS : impact of treatment
Table 1: Impact on VEC Awarness and Activities
![Page 18: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/18.jpg)
No impact on VEC activity No impact on VEC awareness
of additional teacher program
![Page 19: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/19.jpg)
BaselineMean in
ComparisonMean Group Treatment 1 Treatment 2 Treatment 3 Any Treatment
(1) (2) (3) (4) (5) (6)Family A. Dependent Variables - Parental involvment with school Knew about the VEC 0.077 0.040 0.032 *** 0.022 0.023 * 0.026 ***
(0.005) (0.007) (0.012) (0.014) (0.013) (0.010)Could name specific VEC 0.029 0.014 0.024 *** 0.014 0.016 * 0.018 ***members (0.003) (0.004) (0.009) (0.009) (0.008) (0.006)Visited school to monitor or 0.286 0.280 -0.017 -0.040 -0.014 -0.024complain (0.008) (0.015) (0.025) (0.025) (0.025) (0.020)Donated to school 0.065 0.037 -0.001 0.000 -0.006 -0.002
(0.005) (0.007) (0.011) (0.011) (0.010) (0.008)Volunteered at school 0.083 0.040 -0.008 -0.019 ** -0.010 -0.012
(0.005) (0.007) (0.010) (0.009) (0.010) (0.008)Complained about school 0.141 0.092 0.028 0.016 0.019 0.021
(0.007) (0.010) (0.019) (0.018) (0.019) (0.015)Average over family of 0.325 -0.163 -0.019 0.128outcomes (in standard deviations) (0.360) (0.365) (0.396) (0.382)
Family B. Dependent Variables - Parental knowledge of educationSaid don't know when asked how 0.201 0.172 -0.007 -0.044 ** -0.006 -0.018many children can read paragraph (0.008) (0.014) (0.023) (0.020) (0.024) (0.018)Said don't know when asked how 0.213 0.175 -0.012 -0.033 -0.008 -0.017many children can write sentence (0.008) (0.014) (0.023) (0.021) (0.024) (0.018)Perception minus reality of how many 0.123 0.042 -0.014 0.018 -0.040 ** -0.012kids can read paragraphs (0.005) (0.008) (0.018) (0.018) (0.017) (0.014)Perception minus reality of how many 0.109 -0.020 -0.019 0.025 -0.035 * -0.010kids can write sentences (0.004) (0.007) (0.018) (0.018) (0.018) (0.014)Overestimated own child's 0.418 0.336 0.007 0.006 -0.026 -0.005ability to read (0.008) (0.014) (0.022) (0.023) (0.022) (0.018)Overestimated own child's 0.252 0.196 -0.023 -0.003 -0.027 -0.018ability to write (0.007) (0.012) (0.018) (0.018) (0.017) (0.014)Average over family of -0.610 -0.207 -1.262 ** -0.842outcomes (in standard deviations) (0.547) (0.575) (0.556) (0.567)
Family C. Dependent Variables - Prominence of education as a perceived problem in the villageDid the respondent mention 0.131 0.129 0.020 0.012 0.015 0.016education as a problem (0.007) (0.012) (0.021) (0.020) (0.020) (0.016)Was education mentioned 0.033 0.031 0.011 0.016 0.007 0.011at the panchayat (0.003) (0.006) (0.011) (0.011) (0.011) (0.008)Was there any specific meeting 0.009 0.008 0.002 0.001 0.002 0.002on education (0.002) (0.003) (0.005) (0.005) (0.005) (0.004)Average over family of 0.778 0.751 0.624 0.948outcomes (in standard deviations) (0.601) (0.587) (0.618) (0.597)
OLS : impact of treatment
Table 2: Impact on Parents' awareness and activism
![Page 20: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/20.jpg)
Only small impact on parent knowledge (only 5-10% of parents know about the VEC even at endline)
No impact on parent activism
![Page 21: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/21.jpg)
BaselineMean in
comparisonMean group Treatment 1 Treatment 2 Treatment 3 Any Treatment
(1) (2) (3) (4) (5) (6)Faimly B. Dependent Variables - Relationships with parents and parents involvement with schoolHave parents visits the 0.889 0.800 -0.037 -0.034 -0.105 -0.056school (0.018) (0.040) (0.070) (0.067) (0.073) (0.053)Have you organized 0.815 0.720 0.029 0.118 * 0.097 0.082a parents meeting (0.022) (0.045) (0.070) (0.060) (0.070) (0.054)Did parent volunteer 0.358 0.180 0.069 0.045 0.072 0.061in the school (0.027) (0.039) (0.060) (0.060) (0.065) (0.047)Did the school get 0.095 0.030 -0.010 0.009 -0.006 -0.002an allocation from the panchayat (0.017) (0.017) (0.029) (0.029) (0.031) (0.023)Did the school receive 0.060 0.080 -0.042 -0.009 -0.051 -0.033parents' donations (0.013) (0.027) (0.041) (0.047) (0.038) (0.036)Average over family of -0.061 0.471 -0.092 0.158outcomes (in standard deviations) (0.522) (0.460) (0.439) (0.452)
OLS : impact of treatment
Table 3: Impact on school teacher reported parent activism
Non impact on parent activism is confirmed by school teacher responses
![Page 22: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/22.jpg)
No positive impact on school resources
BaselineMean in
comparisonMean group Treatment 1 Treatment 2 Treatment 3 Any Treatment
(1) (2) (3) (4) (5) (6)Family A. Dependent Variables - School resourcesDoes the school 0.959 0.949 0.026 -0.013 -0.013 0.000have textbook (0.011) (0.022) (0.028) (0.034) (0.036) (0.026)Do all the concerned 0.763 0.928 0.049 -0.031 0.054 0.022students get scholarship (0.024) (0.026) (0.048) (0.055) (0.049) (0.042)Does the school have 0.984 0.980 0.002 -0.004 -0.013 -0.005indoor class (0.007) (0.014) (0.008) (0.005) (0.011) (0.007)Does the school 0.924 0.940 -0.063 -0.057 -0.040 -0.054have seat (0.015) (0.024) (0.052) (0.053) (0.044) (0.038)Does the school have 0.940 0.950 0.005 0.002 0.029 0.011have maps and charts (0.013) (0.022) (0.039) (0.039) (0.033) (0.031)Does the school have 0.139 0.100 0.033 -0.005 0.004 0.011a boundary wall (0.020) (0.030) (0.034) (0.047) (0.039) (0.034)Does the school have 0.019 0.060 -0.031 -0.022 -0.014 -0.023electricity (0.008) (0.024) (0.027) (0.030) (0.022) (0.021)Does the school have 0.930 0.980 -0.054 -0.071 * 0.003 -0.043 **water (0.014) (0.014) (0.033) (0.038) (0.020) (0.021)Does the school have 0.456 0.450 -0.016 -0.052 -0.107 * -0.056toilets (0.028) (0.050) (0.061) (0.062) (0.055) (0.048)Does the school have 0.273 0.250 0.006 -0.073 -0.108 * -0.056a toilet for girls (0.025) (0.044) (0.065) (0.062) (0.063) (0.053)Does the school serve 0.687 0.900 0.001 0.059 0.054 0.037midday meal (0.026) (0.030) (0.051) (0.048) (0.047) (0.043)Does the school receive 1.000 0.970 -0.020 -0.066 * -0.056 -0.047 **any governement money (0.000) (0.017) (0.025) (0.037) (0.035) (0.021)Average over family of -0.144 -0.698 ** -0.427 -0.653 **outcomes (in standard deviations) (0.297) (0.315) (0.308) (0.295)
OLS : impact of treatment
Table 3: Impact on school resources (as reported by school teacher)
![Page 23: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/23.jpg)
No positive impact on teacher attendance or effort
BaselineMean in
comparisonMean group Treatment 1 Treatment2 Treatment3 Any Treatment
(1) (2) (3) (4) (5) (6)Family A. Dependent Variables -Teacher number and presence Number of teachers 3.178 4.092 0.127 0.006 0.079 0.069
(0.076) (0.135) (0.162) (0.155) (0.152) (0.125)Number of shiksha mitras 0.621 1.195 0.127 0.230 ** 0.058 0.142
(0.035) (0.080) (0.117) (0.112) (0.117) (0.092)Teacher presence: headmaster's 0.764 0.775 0.023 0.034 0.031 0.030report (0.013) (0.020) (0.037) (0.032) (0.034) (0.027)Teacher presence: Random check 0.753 0.729 0.016 -0.016 -0.004 -0.002
(0.025) (0.028) (0.042) (0.046) (0.049) (0.036)Teacher teaching: Random Check 0.441 0.514 0.013 -0.031 -0.077 -0.030
(0.024) (0.036) (0.058) (0.055) (0.065) (0.047)Regular teacher presence: Random Check 0.719 0.054 0.114 -0.032 0.050
(0.041) (0.103) (0.097) (0.099) (0.078)Regular teacher teaching/ 0.672 -0.002 -0.025 -0.171 * -0.061regular teacher present: Random CheckNot (0.059) (0.090) (0.088) (0.095) (0.076)Shiksha mitra presence: Random CheckAvailable 0.863 0.012 -0.109 0.017 -0.032
(0.047) (0.079) (0.078) (0.079) (0.063)Shiksha mitra teaching/ 0.858 -0.079 -0.072 -0.095 -0.081shiksha mitra present: Random Check (0.045) (0.070) (0.069) (0.073) (0.054)Average over family of 0.296 0.077 -0.287 0.038outcomes (in standard deviations) (0.507) (0.487) (0.536) (0.508)
OLS : impact of treatment
Table 4: Impact on teacher presence and activity
![Page 24: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/24.jpg)
No positive impact on student enrollment and attendance in government school
BaselineMean in
comparisonMean group Treatment 1 Treatment2 Treatment3 Any Treatment
(1) (2) (3) (4) (5) (6)Family B. Depdent Variables - Students enrollment and presence Log(boys enrollment) 4.575 4.522 0.041 0.027 -0.020 0.017
(0.031) (0.060) (0.048) (0.050) (0.069) (0.045)Log(girls enrollment) 4.631 4.636 0.001 0.020 0.013 0.012
(0.030) (0.074) (0.077) (0.074) (0.075) (0.071)Fraction boys present 0.530 0.528 0.029 -0.004 -0.053 -0.008
(0.015) (0.025) (0.041) (0.042) (0.041) (0.032)Fraction girls present 0.496 0.522 0.053 -0.006 -0.027 0.006
(0.014) (0.021) (0.043) (0.035) (0.035) (0.028)Average over family of 0.715 0.138 -0.559 0.128outcomes (in standard deviations) (0.524) (0.507) (0.515) (0.519)
Family C. Dependent Variables - Students Attendance as Reported by ParentsDays present in last 14: all 7.335 6.058 -0.279 -0.599 * -0.314 -0.395children (0.051) (0.104) (0.355) (0.351) (0.371) (0.285)Days present in last 14: only 7.894 6.660 -0.309 -0.564 -0.311 -0.393male children in school (0.065) (0.142) (0.400) (0.387) (0.409) (0.310)Days present in last 14: only 8.139 6.657 -0.215 -0.662 -0.084 -0.316female children in school (0.067) (0.150) (0.388) (0.402) (0.397) (0.309)Average over family of -0.881 -1.670 * -0.547 -1.295outcomes (in standard deviations) (0.937) (0.936) (0.946) (0.938)
OLS : impact of treatment
Table 4: Impact on student enrollment and attendance
![Page 25: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/25.jpg)
BaselineMean in
ComparisonMean Group Treatment 1 Treatment 2 Treatment 3
(1) (2) (3) (4) (5)A. All childrenOut of school 0.069 0.079 0.008 0.006 0.013 **
(0.002) (0.004) (0.005) (0.005) (0.005)In private or NGO school 0.373 0.387 0.009 0.019 -0.006
(0.004) (0.007) (0.016) (0.017) (0.017)Any tutoring 0.069 -0.006 -0.018 ** -0.002
(0.003) (0.009) (0.009) (0.010)Read Class 0.005 -0.001 0.002 0.077 ***
(0.001) (0.002) (0.003) (0.010)
B. Children who were not in school in baselineOut of school 0.758 0.005 0.022 0.001
(0.023) (0.034) (0.033) (0.033)In private or NGO school 0.088 -0.021 -0.030 -0.043 **
(0.015) (0.023) (0.023) (0.021)Any tutoring 0.005 0.004 0.011 -0.006
(0.004) (0.008) (0.008) (0.004)Read Class 0.006 0.007 0.001 0.045 **
(0.004) (0.008) (0.007) (0.018)
C. Children who were in government school in baselineOut of school 0.114 0.013 * 0.003 0.005
(0.006) (0.007) (0.007) (0.007)In private or NGO school 0.150 0.006 0.018 0.005
(0.006) (0.017) (0.019) (0.018)Any tutoring 0.057 -0.009 -0.022 ** -0.006
(0.004) (0.009) (0.008) (0.010)Read Class 0.006 0.000 -0.001 0.094 ***
(0.001) (0.003) (0.003) (0.013)
E. Children who were in a school other than a government school at baselineOut of school 0.025 0.000 0.011 * 0.028 ***
(0.004) (0.006) (0.006) (0.008)In private or NGO school 0.751 0.021 0.024 -0.025
(0.010) (0.024) (0.024) (0.026)Any tutoring 0.087 0.001 -0.011 0.007
(0.006) (0.016) (0.015) (0.017)Read Class 0.003 -0.001 0.007 * 0.048 ***
(0.001) (0.002) (0.004) (0.008)
Table 5: Impact on Chidlren's Schooling Status
OLS : impact of treatment
![Page 26: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/26.jpg)
In “teaching” intervention villages, where reading classes were offered, small increase in children moving out of school, possibly because they were moving into reading camps
![Page 27: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/27.jpg)
Baseline Mean in First stage IV
MeanComparison
GroupTreatment 1 Treatment 2 Treatment 3
attend read class
impact of read class
(1) (2) (3) (4) (5) (6) (7)A. All children Could read at least 0.855 0.892 0.004 0.004 0.017 ** 0.077 *** 0.223 **letters (0.003) (0.005) (0.007) (0.007) (0.007) (0.010) (0.093)Could read at least 0.550 0.635 0.005 -0.003 0.018 ** 0.232 **words or paragraph (0.004) (0.007) (0.008) (0.008) (0.008) (0.101)Could read stories 0.391 0.499 0.004 0.003 0.017 * 0.224
(0.004) (0.007) (0.009) (0.010) (0.010) (0.137)
B. Children who could not read at Baseline Could read at least 0.432 0.041 0.032 0.079 ** 0.131 *** 0.602 **letters (0.019) (0.031) (0.034) (0.035) (0.023) (0.304)Could read at least 0.056 -0.006 -0.013 -0.007 -0.051words or paragraph (0.009) (0.015) (0.012) (0.014) (0.106)Could read stories 0.028 -0.006 -0.013 -0.008 -0.063
(0.006) (0.010) (0.008) (0.009) (0.074)
C. Children who could only read letters at Baseline Could read at least 0.919 -0.008 -0.015 0.021 0.132 *** 0.162 *letters (0.008) (0.016) (0.014) (0.013) (0.020) (0.097)Could read at least 0.253 -0.011 -0.025 0.035 0.269words or paragraph (0.013) (0.022) (0.021) (0.022) (0.171)Could read stories 0.086 -0.001 -0.010 0.033 ** 0.261 *
(0.009) (0.014) (0.014) (0.017) (0.135)
D. Children who could read up to words or paragraph at baseline Could read at least 0.988 -0.001 0.006 0.006 0.074 *** 0.068letters (0.003) (0.006) (0.004) (0.004) (0.012) (0.065)Could read at least 0.813 0.032 * 0.010 0.044 ** 0.614 **words or paragraph (0.012) (0.019) (0.019) (0.017) (0.271)Could read stories 0.520 0.010 0.010 0.032 0.458
(0.015) (0.026) (0.025) (0.027) (0.388)
E. Children who could read a story at baseline Could read at least 0.994 0.001 0.004 * -0.001 0.030 *** -0.058letters (0.002) (0.003) (0.002) (0.003) (0.006) (0.088)Could read at least 0.973 0.004 0.008 0.004 0.116words or paragraph (0.004) (0.006) (0.005) (0.005) (0.170)Could read stories 0.909 0.006 0.012 0.007 0.234
(0.007) (0.011) (0.012) (0.011) (0.350)
F. Children who could only read letters, words, or paragraph at baselineCould read at least 0.954 -0.004 -0.005 0.014 ** 0.102 *** 0.132 *letters (0.004) (0.009) (0.008) (0.007) (0.014) (0.068)Could read at least 0.540 0.011 -0.008 0.039 *** 0.395 **words or paragraph (0.011) (0.014) (0.016) (0.015) (0.160)Could read stories 0.309 0.006 0.000 0.034 * 0.343 *
(0.010) (0.016) (0.016) (0.017) (0.185)
Table 6: Impact on Reading
OLS : impact of treatment
![Page 28: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/28.jpg)
Improvement over time among illiterate children
Endline levels: Children reading nothing at baseline
-20%0%
20%40%60%80%
100%120%
Con
trol
trea
tmen
t 1
trea
tmen
t2
trea
tmen
t 3
Afte
r re
adcl
ass
(est
imat
ed)
Letter
word or para graph
Story
![Page 29: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/29.jpg)
Improvement over time among children who could recognize letters
Endline level: Children who could read letters at baseline
0%
10%
20%
30%
40%
50%
60%
Co
ntr
ol
tre
atm
en
t2
Aft
er
rea
d
cla
ss
word or paragraph
Story
![Page 30: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/30.jpg)
Improvement over time among children who could read words or paragraphs
Endline level: Children reading words or paragraph at baseline
0%20%40%60%80%
100%120%140%
Co
ntr
ol
tre
atm
en
t2
Aft
er
rea
d
cla
ss
word or paragraph
Story
![Page 31: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/31.jpg)
Summary of Impact No impact at all on public schools, or on VEC activism
Learning improved because an offer to the community of a teaching program was taken up by local youth volunteers—reading classes were held
Children who enrolled in these classes made significant improvements in reading within a couple of months
No evidence, anecdotal or otherwise, of VECs, or Village Heads, or school teachers, supporting these volunteer-led reading classes, or mainstreaming the reading program in public schools
![Page 32: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/32.jpg)
Why was there no impact on public schools?
Not because of lack of interest in education, or willingness to participate, or because it’s just too difficult to get children to learn
Contrast between impact outside versus within the public school system suggests that the real challenge lies in improving public provision
![Page 33: Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India](https://reader036.vdocuments.us/reader036/viewer/2022062422/56813d5c550346895da726b6/html5/thumbnails/33.jpg)
Conclusions Significant barriers to participation, for
social accountability of public providers
Instituting participation from outside, in lagging or disadvantaged areas, likely requires credible support from above—from government or influential NGOs
“Short route” to accountability (WDR 2004) likely requires going a long distance