OUT-OF-SCHOOL TIMEAS A DEVELOPMENTAL CONTEXT

Lessons from Successful (and Unsuccessful Afterschool Programs

Deborah Lowe VandellSRHD Biennial Conference

San Antonio TXMarch 2010

The Policy Context• Disconnect between the length of the school day and children’s

academic, social, behavioral, and physical needs

• High rates of maternal employment– 74% of the mothers of children 6-17 yrs

• Concerns about– Low test scores– Unmet needs of English Language Learners– Negative effects of low supervision– Youth as victims & perpetrators of crime – Childhood obesity

• Some evidence of beneficial effects of afterschool programs but not always

• An element in Race to the Top and other school reform efforts

What is an after-school program?

• Narrow definition – programs that are offered by schools or other organizations on a daily basis throughout the school year

• Broad definition – includes extracurricular activities, sports, clubs, and sports offered by schools, libraries, and youth organizations

After-School Programs Sometimes Offer

Academic enrichment & remediation• Homework help• Science, computer & math clubs• Book clubs

Non-academic enrichment• Organized sports & recreational games• Music, drama• Arts and crafts• Scouts, 4-H, YMCA

Public and Private Investments in Programs

• After-school programs– Serve 7 million+ children– 21% of 6 to 9 yr olds & 14% of 10-12 yr olds– CA After-School Education and Safety Program– 21st Century Community Learning Centers– National Network of State Afterschool Networks

• Participation in at least 1 organized activity in a yr – 81% of 6- to 11-yr-olds– 83% of 12- to 17-yr-olds– 90% of non-poor vs 60% of poor children

Do afterschool programs have positive effects on child

developmental outcomes?

Effect Sizea statistic that measures the magnitude of a program’s impact on a particular outcome

One common metric – Cohen (1988)“small” - d = .2"medium” - d = .5“large” - d = .8

Effect sizes also can be benchmarked against those reported in other studies.

1. Aspirin on heart disease d = .03 2. School-based substance abuse prevention programs on

drug & alcohol use d = .093. Class size reductions on math achievement d = .23

A RECENT META-ANALYSIS

Durlak, J. A., & Weissberg, R. P. (2007)

49 reports of 73 programs

Meta-analyses of 1. All programs

2. Programs with Sequential & Active activities and Focused & Explicit content (SAFE)

• .

Durlak & Weissberg Meta-AnalysisOutcomes # of

StudiesOverall Effect Size

Met SAFE Criteria

Did not meet SAFE criteria

Self-perceptions

22 .34 .35 .14

School bonding

28 .14 .26 .03

+ social behaviors

35 .19 .30 .06

Misconduct 42 .18 .26 .07

Drug use 27 .11 .22 .03

School Achievement

20 .16 .31 .03

Grades 25 .11 .24 .05

School Attendance

20 .16 .31 .03

ADDITIONAL COMPONENTS OF PROGRAM QUALITY (NRC, Eccles & Gootman, 2002)

• Positive relationships between students and staff

• Positive relationships between students

• Mix of academic and non-academic skill-building activities

• High levels of student engagement• Mastery orientation• Appropriate levels of structure• Opportunities for autonomy and

choice

Study of Promising Afterschool Programs(Vandell, Reisner, Pierce et al. 2007)

• A longitudinal study of almost 3,000 low-income, ethnically diverse elementary and middle school student

• Eight states: CA, CO, CT, MI, MT, NY, OR, RI

• 14 communities: rural, small towns, mid-size cities, large cities

• All located in high-poverty communities

• First year devoted to identifying & describing promising programs

• Studied student academic & social outcomes over

two years

Programs received consistently high ratings on all of the program quality dimensions

• Positive staff-student relationships • Positive relationships between

students• Mix of skill-building activities• High levels of student engagement• Mastery orientation• Appropriate levels of structure• Opportunities for autonomy and

choice

Study ParticipantsElementary School Sample

1,796 students in Grades 3 & 4 from 19 schools89% free- or reduced-price school lunch88% students of color

Middle School Sample1,118 students in Grades 6 & 7 from 16 schools

63% free- or reduced-price lunch69% students of color

• Organized activities such as team sports, school-based activities and lessons

• Home alone and home with a sibling• “Hanging out” with peers without adult supervision

• Each reported on 4-point scales– 1 = not at all/ once or twice– 2 = about once a week– 3 = 2 – 3 times a week– 4 = 4 or more days a week

Other Afterschool Experiences

– Programs only (PO)– Programs plus other activities (PP)– Low supervision with activities– Other

FOUR CLUSTERS WERE IDENTIFIED

Participation in Promising Programs and Other Activities

Over 2 YearsElementary Sample • 54% regular program participants (about

90 day/yr)2/3 Program Only; 1/3 Program Plus

• 15% low supervisionMiddle School Sample• 49% regular program participants (about

55 days/yr)2/3 Program Only; 1/3 Program Plus

• 16% low supervision

Child OutcomesMeasured at Baseline, at the end of

Year 1, and at the end of Year 2

Test Scores Self-Reports Teacher Reports

MathReading

Work habitsMisconductSubstance use

Work habitsTask persistenceAcademic performanceProsocial behaviorAggression

• HLM analyses • Level 1 - child• Level 2 – school/program• Difference scores • 10 multiple imputations

Analytic Plan

Covariates

• Gender • Race-ethnicity • Grade level• Household income (per $1000)• Maternal (or guardian) education (in yrs)• Mother (or guardian) works full time• Household structure

Elementary Sample: Child Report Program Only vs. Low Supervision

Effect Sizes

-.58

.17

-.66

0.24

-0.70

-0.60

-0.50

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

Work habits Misconduct

Year 1Year 2

Elementary Sample: Child Report Program Plus vs. Low Supervision

Effect Sizes

-.43

.36

-.51

.41

-0.60

-0.50

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

Work habits MisconductYear 1Year 2

Elementary Sample: Teacher Report Program Only vs. Low Supervision

Effect Sizes

-.17

.19

.24 .23

.18 .17

-.34

.23.21

.23

.31

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

Academic Work habits Taskpersistence

Social skills Prosocialbehavior

Aggression

Year 1Year 2

Elementary Sample: Teacher Report Program Plus vs. Low Supervision

Effect Sizes

-.29

.21

.30.30

.35

0.25

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

Academic Work habits Taskpersistence

Social skills Prosocialbehavior

Aggression

Year 1Year 2

Elementary Sample Math Achievement Test Scores

Effect Sizes

.61

.73

.52

00.10.20.30.40.50.60.70.8

Program Only Program Plus

Year 1Year 2

Middle School Sample: Youth ReportProgram Only vs. Low Supervision

Significant Effect Sizes

-.32 -.31

-.47

-.55

0.20

-0.60

-0.50

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

Work habits Misconduct Substance use

Year 1Year 2

Middle School Sample: Youth Report Program Plus vs. Low Supervision

Effect Sizes

-.37-.31

0.33

-.64-.67-0.70

-0.60

-0.50

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

Work habits Misconduct Substance use

Year 1Year 2

Middle School Sample Math Achievement Test Scores

Effect Sizes

.57.55

00.10.20.30.40.50.60.70.8

Program Only Program Plus

Year 1Year 2

Conclusions: Elementary School Sample

Standardized TestsGains in math for PO after one year; gains for both PO and PP groups after 2 years

Child Reports Gains in work habits & reductions in misconduct for both PO & PP groups; larger effects after 2 years

Teacher ReportsGains in work habits, task persistence, social skills, prosocial behaviors & reductions in aggression for PO after first year; improvements for both PO and PP after 2 years

Conclusions: Middle School Sample

Standardized TestsGains in math achievement for PO & PP groups after 2 years (not 1 year)

Youth Self-Reports Both PO and PP youth reports reported gains in work habits after 2 years (not 1 year) and larger reductions in misconduct and substance use after two years

Teacher ReportsLittle evidence of differences between Program and low supervised youth

Why are after-school programs beneficial?

Study 2: Experience Sampling - Out-of-School Time

Vandell, Shernoff, et al. (2007, 2009) • 191 8th grade students, primarily low-income students

of color

• 8 middle schools in 3 communities • Students wore watches that beeped them at random

times during non-school hours – afterschool, evenings, weekends

• 35 signals during a one week period in the fall and 35 signals during a one week period in the spring

Students Filled Out Logbooks

• After each signal, students recorded –Who they were with–Where they were –What they were doing–How they were feeling–And their levels of effort, concentration,

motivation

Very Little Missing Data!!

• On average, students responded to 33 of the 35 signals during the week.

• 12,143 after-school, evening, and weekend experiences were reported.

• 5, 136 of the experiences occurred after school.

Students’ Activities at Programs and Elsewhere

Student Activities

0 5 10 15 20 25 30 35

Homework

Enrichment

Eating

TV

Sports

Service

% of time

At programNot at programNo program

Differences in Supervision (% of time)

No program

Not at program

At Program

Unsuper’d peers

16.7 21.0 7.2***

Sup. Peers 26.4 26.0 91.1***No adults 38.4 38.4 7.5***Alone 12.9 11.7 0.0Sib care 5.8 3.5 0.1

Differences in Motivation, Effort, and Feelings (4-pt ratings)

No program

Not at program

At program

Motivation 2.7 2.9 3.0***Effort 1.9 1.9 2.5***Importance

2.4 2.5 2.9***

Positive Emotions

2.2 2.3 2.5***

Negative Emotions

1.3 1.2 1.3

Choice, Concentration, and Effort

Study 3: Are Particular Program Features Associated with Child Developmental

Outcomes? (Pierce, Bolt, Vandell, 2010)

3-year longitudinal study of children who attended 30 programs of varying quality

n = 150 in 1st grade n = 120 in 2nd grade

n = 91 in 3rd grade

49% male89% white60% parents have college degrees

Observations Of Program Quality

4 program observations in 2nd grade and 3 observations in 3rd grade

staff-child relationships availability of age-appropriate activities

programming flexibility

Analyses control for family background and child prior functioning

Child Developmental Outcomes

• Teacher ratings – Mock Report Card

• Reading• Math

– Work habits (6 items, 5 pt ratings)– Social skills with peers (7 items, 5 pt ratings

• Collected from classroom teachers in G1, G2, and G3.

Analytic Strategy

HLM analyses in Grade 2 & Grade 3Level 1 – childLevel 2 – program

Covariates:gender, minority (yes/no), maternal education, family income, 2-parent household (yes/no), firm/responsive parenting, prior child functioning ,

Effects Associated With Program Quality Features in G2 and G3

Five Take-Home Messages • Regular participation in high quality afterschool programs is linked to

positive social and academic outcomes

• Gains are more evident after two years than after one year, suggesting that duration is important

• Youth reports of experiences reveal differences in activities, social partners, motivation, effort, and affect while attending afterschool programs

• There is evidence that particular aspects of children’s experiences are related to child developmental outcomes

• Out of school time IS an important developmental context

http://childcare.gse.uci.edu/

for more information