Massachusetts Universal Pre-Kindergarten Program

Evaluation of the First Two Years of the Pilot Initiative

Alyssa Rulf FountainBarbara Goodson

September 2008

Today’s Presentation

1. Overview of evaluation

2. Key findings

3. Policy implications

4. Next steps for evaluation

05/03/23 2

Massachusetts UPK

• Goal of UPK: to promote school readiness and positive outcomes for children, especially at risk

• Pathway to outcomes: through universally accessible, quality early education and care

• UPK being enacted through mixed service delivery system

05/03/23 3

UPK Pilot Initiative: Implemented FY 2007

• Grants awarded to selected eligible providers to enhance quality of care

• Grants were targeted to settings that already showed commitment to quality

• Grantees in the evaluation received grants in both FY07 and FY08

05/03/23 4

UPK Classroom Quality Grants: FY07 and FY08• Evaluation includes the 68 agencies and 125

program sites (centers, public school preschool programs, and family child care providers) that received Classroom Quality grants in FY07– Child care centers: 52 agencies/81 centers

– School districts: 5 school district program sites

– Family child care: 9 agencies/39 providers

• In FY08, an additional 87 program sites received Classroom Quality grants (not included in this evaluation)

05/03/23 5

Evaluation of UPK Pilot Program

• Current evaluation represents first step in assessing UPK

– Focus on implementation process

– Measurement of most immediate outcomes—use of funds, perception of impacts on quality

• Next steps for evaluation– Measuring longer-term outcomes on quality

and child outcomes

05/03/23 6

• Research questions– Characteristics of grant recipients and children they

serve

– How grantees allocated grant funding across allowable expenditure areas

– Grantee/provider satisfaction with grant program

– Areas where greatest program needs remain

• Interviewed all program administrators and a sample of teachers and providers

Summary of Evaluation Design

05/03/23 7

Summary Findings: Implementation of Pilot initiative• Pilot grant program being implemented as

planned

• Agency satisfaction with grant program

–High marks for targeting of grant funds to areas linked to quality

–High marks on perceived positive effects of grants on quality of care

05/03/23 8

Abt Associates Inc.

Allocation of UPK Grants by Year—Types of Expenditures for All Grantees

88

98

22

81 79

52

19

31

9295

62

87 89

59

23

82

0

10

20

30

40

50

60

70

80

90

100

Assessment Curriculum/ed.materials

Accreditation Staffcompensation

P rofessionaldevelopment

Comprehensiveservices

Full-day/full-year services

Administrativecosts

%

20072008

UPK Grant Fund Allocations: Any Grant Funds by Year

Types of Expenditures by Grantee Type

96 96

67

90 92

61

86

75 75 75 75 75 75

25

50

100

25

38

13

75

25

757575

0

10

20

30

40

50

60

70

80

90

100

Assessment Curric./ed. Mat.

Accreditation Staff comp.

PD Comp.serv.

Full-day/full-year

Admin. costs

%

Center-based grantees Public school grantees family child care grantees

UPK Grant Fund Allocations: Any Grant Funds by Grantee Type (2008 only)

Abt Associates Inc.

Allocation of UPK Grants by Year—% of Fund Expenditures for All Grantees

16

29

1

24

9 63 3

1014

4

31

16

92

6

0

10

20

30

40

50

Assess

ment

Curricu

lum/ed. m

ateria

ls

Accre

ditati

on

Staff c

ompe

nsatio

n

Profess

ional

deve

lopmen

t

Compr

ehens

ive se

rvice

s

Full-d

ay/fu

ll-yea

r ser

vices

Admini

strati

ve co

sts

%

20072008

UPK Grant Fund Allocation: % Grant Funds by Year

Abt Associates Inc.

% Expenditures by Grantee Type

13

4

32

1717

6 5

44

4

13

6 47

27

5

20

40

622

10 10

20

0

10

20

30

40

50

Assessment Curric./ed. Mat.

Accreditation Staff comp.

PD Comp. serv.

Full-day/full-year

Admin. costs

%

Center-based grantees Public school grantees family child care grantees

UPK Grant Fund Allocation: % Grant Funds by Grantee Type (2008 only)

Summary Findings: Presence of At-Risk Children in Grantee Program Sites• Sixty four percent of children receiving financial

assistance

• Half of children from minority groups

• 1/3 of children from non-English language homes

• 15% of children identified with special needs

• Proportion at-risk children (low income, minority language/cultural groups) varies by type of care, with more at-risk children in family child care

05/03/23 13

Abt Associates Inc.

Race/Ethnicity of Children in UPK Programs—All Grantees

Abt Associates Inc.

Race/Ethnicity of Children in UPK Programs—All Grantees

51%

25%

16%

8% WhiteHispanicBlack, non-HispanicOther

Children in All Programs

Abt Associates Inc.

Race/Ethnicity of Children in UPK Programs by Grantee Type

Abt Associates Inc.

Race/Ethnicity of Children in UPK Programs by Grantee Type

50%

23%

18%

9%

49%

39%

9% 3%60%19%

11%

10%

Children in Center-based Programs

Children in Family Child Care Homes

WhiteHispanicBlack, non-HispanicOther

Children in Public School Programs

Abt Associates Inc.

• Classroom teachersAverage = 14 years early childhood experience 18% had an associate’s degree 6% had a CDA 47% had a bachelor’s degree 29% had a graduate degree

• Family child care providersAverage = 18 years of early childhood experience 8% had an associate’s degree 67% had a CDA 25% had a bachelor’s degree

UPK Teacher/Provider Backgrounds

Grantee Views of Remaining Needs (Beyond Current Grant Funding)• Program sites

– Most pressing need identified by majority of grantees: Staff compensation

– Half of child care center programs also see need for professional development

• Family child care agencies– Wider range of needs identified– Needs identified by more than half of agencies:

professional development, comprehensive services (60%)

– Half of agencies also see need for curriculum/ educational materials and staff compensation

05/03/23 17

Overview of Policy Implications

• Need strategies to improve quality of care for all program sites, regardless of starting point

• Develop strategies for promoting quality in different types of care settings

• Refine allowable uses of grant funds, especially by same sites over time

• Bring parents into the equation

05/03/23 18

Policy Implications: Strategies to Improve Quality of Care• Need two-pronged approach

– Research about effecting child outcomes supports targeting quality to achieve excellence

– Need simultaneous support for lower quality programs

• Need to consider providing more quality-related technical assistance/training to ensure maximum impact on quality--programs can’t do it alone

05/03/23 19

Abt Associates Inc.

Policy Implications: Improving Quality in Different Types of Care Settings• Family child care grantees present unique challenges

Agencies focus on materials even with time for planning

Providers more removed from grant programHigh priority because of presence of high % of at-risk

children/children from minority language and cultural groups in family child care

• Public school programs present different challenges– School districts serve a greater number of children

with special needs

– Perceive a need for longer days with children

05/03/23 20

Abt Associates Inc.

Policy Implications: Guidance on Use of Grant Funds

• Possible changes in allowable areas of spending to target areas with closest links to child outcomes

– Consider becoming more prescriptive/ narrowing allowable fund uses

– Possibility of tightening spending uses over time, leaving broader in first years and narrowing allowable uses in later years

05/03/23 21

Abt Associates Inc.

Policy Implications: Bringing in Parents

• UPK program currently low-profile for public

• Parents need to play a more critical role as the program grows

– Programs need help with parent materials to explain program and its relevance to families, children

– Parent choices still mostly driven by personal considerations (cost, location, hours, etc). Quality considerations often secondary.

05/03/23 22

Abt Associates Inc.

Next Steps in Evaluation: Assessing Program Quality• Looking at longer-term outcomes for early

care and education system

• Level of quality of care in all programs and separately for UPK, UPK-eligible settings

– Directly measure program quality in sample of including all types of settings

• Accessibility of quality care for all children but especially at-risk children

– Examine, for sample of communities, the supply of UPK and UPK-eligible care settings compared with demand

05/03/23 23

Abt Associates Inc.

Next Steps in Evaluation: Descriptive Study of Child Outcomes

• Assess skills in early literacy, math, social adaptation, self-regulation

• Assess children in random sample of settings

• Stratify by type of care settings, UPK status• Assess at multiple points in time to create

longitudinal picture over time–Beginning & end of Pre-K, end of kindergarten

& grade 1

• Continue assessments over multiple years, to establish trends over time

05/03/23 24

Abt Associates Inc.

Measuring Child Outcomes: What This Can Tell Us• Status of MA children over time vs national

norms

• Status of subgroups of children–Children in UPK settings, UPK-eligible

settings, other settings–Children in different types of settings–At-risk children

05/03/23 25