evaluating how tayari’s program impacts › public... · 2018-04-09 · tayari longitudinal study...

13
Persistence of pre-primary programs in Kenya: Evaluating how Tayari’s program impacts change over time and into primary school Dr. Benjamin Piper March 28, 2018

Upload: others

Post on 30-Jun-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Persistence of pre-primary programs in Kenya:

Evaluating how Tayari’s program impacts

change over time and into primary school

Dr. Benjamin Piper

March 28, 2018

Page 2: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Tayari Counties• Laikipia• Uasin Gishu• Siaya• Nairobi• Nairobi APBET

Page 3: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Introduction to Tayari

Pilot program at medium scale (2000+ centers) from 2016-18

Implemented by four counties with technical support from RTI

RCT research design

Three treatment groups

Tayari longitudinal endline study on treatment 2

2016 cohort representative of 600+ centres

Page 4: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Costs per Treatment Group

$1.70 $2.50 $2.50

$4.12$4.97 $4.67

$1.65

$1.43 $2.69$0.70

$0.61$0.60

$0.00

$2.43$2.43

$0.00

$0.20$0.21

$0.00

$0.00

$1.85

$0.00

$2.00

$4.00

$6.00

$8.00

$10.00

$12.00

$14.00

$16.00

Treatment 1 Treatment 2 Treatment 3

SCECDC training Teacher Training Coaching

Other materials Learner Workbooks Teachers' Guides

Health Training

Page 5: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Tayari Average Impact (Effect sizes)(APHRC, 2018)

0.19

0.10

0.29

0.43

0.20

0.68

0.37

0.22

0.53

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

Overall Phase 1 Phase 2

T1 T2 T3

Page 6: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Tayari Longitudinal Study by Cohort and Time

CohortJan 2016

baseline 2016 academic yearOct 2016

midterm 2017 academic yearOct 2017

endlineTotal 3,257 11.2% attrition 2,891 18.7% total attrition 2,647

91.7% in same school

as Jan

91.5% in same

school as last year

PP1 cohort 1,126 82.4% still in PP1 1,010 81.8% in PP2 930

93.4% in same school as Jan

92.8% in same school as last year

PP2 cohort 1,365 82.1% still in PP2 1,227 90.4% in Grade 1 1,128

88.4% in same school as Jan

92.1% in same school as last year

Combined cohort

766 91.5% still in Combined

654 8.7% in Combined 589

95.5% in same school as Jan

88.6% in same school as last year

Page 7: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Distribution of Tayari Literacy Scores

0%

2%

4%

6%

8%

10%

12%

14%

PP1 PP2 Combined

Page 8: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Distribution of Tayari Numeracy Scores

0%

2%

4%

6%

8%

10%

12%

14%

PP1 PP2 Combined

Page 9: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Impact on Tayari Indices

PP1 Cohort – 2 years PP2 Cohort – 1 year

Difference p-value

Effect size (SD)

Difference p-value

Effect size (SD)

School Readiness

Index (longitudinal)

6.48 .01 0.38 1.42 .43 0.07

School Readiness

Index (external)

4.97 .05 0.27 0.13 .95 0.01

Literacy index 6.51 .02 0.31 –0.78 .73 –0.04

Numeracy index 8.56 .03 0.25 3.28 .18 0.16

Socio-emotional

and executive function index

4.13 .13 0.20 –3.62 .24 –0.18

Page 10: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Impact of 1 year of Tayari on Grade 1

Subject

With controlsWith controls and baseline

performance

Differencep-

valueEffect Size Difference p-value

Effect size

EnglishLetter sounds (per minute) –3.86 .21 0.17 –3.68 .20 0.16

Oral reading fluency (per minute) –6.30 .11 0.24 –6.06 .11 0.23

Reading comprehension (% correct) –5.76 .28 0.18 –5.46 .29 0.17

KiswahiliLetter sounds (per minute) –4.65 .12 0.23 –4.49 .10 0.23

Syllable sounds (per minute) –6.46 .01 0.30 –6.27 .00 0.29

Oral reading fluency (per minute) –4.99 .01 0.31 –4.86 .00 0.30

Reading comprehension (% correct) –7.06 .01 0.31 –6.87 .01 0.30

MathematicsNumber identification (per minute) –4.11 .02 0.28 –2.91 .05 0.20

Addition fluency (per minute) –0.38 .36 0.10 –0.07 .84 0.02

Subtraction fluency (per minute) –0.54 .20 0.17 –0.27 .48 0.08

Quantity comparison (% correct) –1.49 .64 0.05 0.60 .83 0.02

Page 11: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Gains at Midterm and Endline(Effect Sizes in SD)

0.31 0.32

0.45

0.38

0.07

0.30

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

PP1 PP2 Combined

Midterm Endline

Page 12: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Reflections

• Pre-primary fadeout effect is typical

• 1 year of Tayari insufficient to resist fadeout

• October 2018 results will show whether 2 years of Tayari resist fadeout

• Nil effects of 1 year of Tayari on EGRA/EGMA

• Kenya’s Competency Based Curriculum

• Angst about “academic” pre-primary

• Revisions to Tayari based on findings

Page 13: Evaluating how Tayari’s program impacts › public... · 2018-04-09 · Tayari Longitudinal Study by Cohort and Time Cohort Jan 2016 baseline 2016 academic year Oct 2016 midterm

Thank you

[email protected]