1718wccusd finalreport v4 · harold dichoso administrative technician public profit evaluation team...
TRANSCRIPT
EXPANDED LEARNING West Contra Costa Unified School District
2017-2018 Annual Findings Report
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
2
ACKNOWLEDGEMENTS
The WCCUSD Expanded Learning Program Evaluation requires active participation from many partners
throughout the year. We would like to express our gratitude to the students, families, and Expanded
Learning Program staff for contributing to this evaluation and report.
WCCUSD EXPANDED LEARNING PROGRAMS OFFICE
KATHARINE SULLIVAN
Expanded Learning Coordinator
ALICIA AZCARRAGA
Program Assistant
HAROLD DICHOSO
Administrative Technician
PUBLIC PROFIT EVALUATION TEAM
STEPHANIE KONG
Research Associate
HANNAH PICKAR
Research Assistant
DA’SHON CARR
Project Assistant
JOSEPH MONARDO
Graduate Intern
COREY NEWHOUSE
Founder and Principal
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
3
WCCUSD EXPANDED LEARNING 2017-2018
ABOUT EXPANDED LEARNING PROGRAMS IN WCCUSD
28 Programs
3 Lead Agencies
$5.6M In Partner Investments
3,484 Youth Participants
Programs served a higher proportion of Latino and African American students than their host schools in the District, the County, and the State.
47% of particpants were English Language Learners
Programs served a high proportion (47%) of English Language Learners. ELL participants who attended 100 days or more of program were significantly more likely to be reclassified (13%) than their peers who attended fewer than 100 days (9%).
Participants attended 112 days of program, on average. Participants also attended the school day at a higher rate than non-participants (95% and 93%, respectively) and were less likely to be chronically absent from school than their non-participant peers.
!
Latino 60%
African American
23%
Asian 10%
White6%
Other1%
120
68
Overall Average
days attended
112
ES MS
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
4
FOCUS ON QUALITY
•! Overall, program quality improved across sites at both the elementary and middle
school levels from fall to spring, as demonstrated by higher program quality assessment
scores. In both fall and spring, elementary and middle school programs scored greater
than 4.50 in the safety domain which indicates that Expanded Learning Programs were
particularly strong when it comes to providing an emotionally and physically safe
environment for their youth. Middle school sites in particular increased scores in the
engagement domain over the course of the year.
•! Corroborating evidence from survey data demonstrated that youth and their families feel
their Expanded Learning Programs provide a very safe and supportive environment.
Furthermore, youth reported increases in engagement compared to the previous year.
The majority of youth reported that the adults in the program listen to what they have to
say (83%), an increase from last year (77% in 2016-17). While less than half of youth felt
they get to choose what they do or how they do it during program activities (48%), this
was an improvement over last year when just 37% of youth agreed.
SCHOOL DAY AND DISTRICT GOAL ALIGNMENT
•! The majority of Expanded Learning Programs have strong relationships with their host
schools and are willing to cooperate with one another. However, only 57% of staff
reported that the program’s curriculum and instruction reinforce concepts taught during
the school day.
•! Participants that attended 100 days or more of program were significantly more likely
to be reclassified as English proficient in 2017-18 (13%) than their peers that attended
less than 100 days (9%).
•! Participants attended an average of 95% of school days enrolled, compared to an average
of 93% among non-participants, which translates to approximately four more days of
academic instruction. Related, program participants were significantly less likely than
their non-participant peers to be chronically absent from school. In particular,
Black/African American participants have a relatively high chronic absence rate, but the
chronic absence rate for Black/African American participants was significantly lower
than for Black/African American non-participants (25% compared to 39%, respectively). !
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
5
TABLE OF CONTENTS
About Expanded Learning in WCCUSD 6
WCCUSD Expanded Learning Participants 13
Expanded Learning Programs Support School Day and District Goals 21
Recommendations 38
Appendices 40
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
6
ABOUT EXPANDED LEARNING IN WCCUSD WCCUSD EXPANDED LEARNING VISION
Youth in West Contra Costa Unified School District (WCCUSD) have access to positive youth
development experiences that help youth thrive and become successful in school and
beyond. In order for youth to thrive, they need to feel safe, have positive relationships
with caring adults, feel that they belong, and experience appropriate and engaging
challenges aligned with their interests. Expanded learning programs can provide these
very elements for youth in the critical hours after school.1
WCCUSD’s Expanded Learning Office promotes program quality using a continuous quality
(CQI) improvement framework, seeking to ensure that positive youth development
practices are driving success for youth in and out of the classroom. WCCUSD Expanded
Learning Programs are open to all students2 at 28 schools at no cost to families.3
Community Youth who live in under-resourced communities, who may be living in poverty, or for
whom English is not their first language, may face barriers to academic achievement and
school success. These are the students most in need of high quality developmental
experiences.4 WCCUSD is one of several school districts within Contra Costa County and
the gaps between the District and the County communities are noticeable. The majority of
WCCUSD students are eligible for free and reduced-priced meals (70%), far above Contra
Costa County as a whole (39%).5 Furthermore, 34% of students in WCCUSD are English
Language Learners, far more than the County (17%).6 Given these discrepancies, it is clear
that the youth in WCCUSD are in need of additional supports to ensure they have
opportunities and experiences that place them on track with their peers in the broader
community.
1 Gambone, M.A., Klem, A.M., and Connell, J.P. (2002). Finding Out What Matters for Youth: Testing Key Links in a Community Action Framework for Youth Development. Philadelphia: Youth Development Strategies, Inc., and Institute for Research and Reform in Education. 2 Host schools determine specific criteria for priority student enrollment, such as low academic performance or social needs. 3 Per grant legislation, school-based 21st Century and After School Education and Safety programs may charge a fee but may not turn away youth for inability to pay. 4 Afterschool Alliance (2016). America After 3PM Special Report: Afterschool in Communities of Concentrated Poverty. 5 Education Data Partnership, 2016-17 Student Demographics for Contra Costa County and West Contra Costa County Unified School District. 6 Ibid.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
7
Partners WCCUSD contracts with community organizations that have expertise in youth
development and out of school time programming to operate its 28 ASES/21st Century after
school programs. These community-based organizations work in close partnership with the
WCCUSD Expanded Learning Office and the school community to design and implement
high quality programs for WCCUSD students. Each Expanded Learning Program offers
academic support, enrichment activities, and hot supper five days per week throughout
the school year. In addition to its three lead partners, Bay Area Community Resources
(BACR), Love. Learn. Success. (LLS), and the YMCA of the East Bay, the Expanded Learning
Office convenes quarterly collaborative meetings with all stakeholders to inform and
coordinate services and efforts across the Expanded Learning Programs in WCCUSD. The
goal of the collaborative meetings is to share the program’s mission, vision, goals, and
program outcome data. These collaborative partners include leadership from local
government, philanthropy, and organizations who specialize in visual and performing arts,
mentoring, sports, and STEAM programming.
Funding The District’s Expanded Learning Programs are funded through a mix of state and federal
dollars, supported by a local match of funding from each of the lead agencies and
additional community partners. All 28 sites are funded by the state’s After School
Education and Safety (ASES) grant. One elementary school site was also funded by the 21st
Century Community Learning Center federal grant to provide school year and summer
programming. Through the combination of these funding streams, along with community
partners, WCCUSD continued to make a significant financial investment in its Expanded
Learning Programs (Figure 1).
Figure 1. WCCUSD and partners invest $5.6 million in expanded learning
State:
ASES
$3,711,260
Federal: 21st
CCLC
$122,850
Community
Agencies
$1,484,084
WCCUSD
$XXXX
$5,620,694
Total Investment
$302,500
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
8
EVALUATION APPROACH
The WCCUSD Expanded Learning Office continued its partnership with Public Profit to
evaluate its Expanded Learning Programs in 2017-18. We used a mixed-methods approach
to the evaluation, collecting data on program participation, program quality, school day
alignment, and program effectiveness through the District, surveys, and observations, in
alignment with the District’s theory of action. This report contains the results of the 2017-
18 evaluation and includes information about:
•! WCCUSD’s commitment to quality Expanded Learning Programs and program staff;
•! Program participation and attendance;
•! Stakeholder reports of program quality; and
•! Alignment with the school day and District goals.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
9
INVESTMENT IN QUALITY
Participation in an expanded learning program can lead to positive outcomes for youth,
however, program quality is a critical hinge in that pathway. Research suggests that
regular participation in high quality programs is correlated with improved grades, school
day attendance, study habits, task completion, and communication skills.7 The WCCUSD
Expanded Learning Office continued to define and measure program quality using the
Weikart Center for Youth Program Quality framework. Understanding program quality is
the key to youth’s subsequent success. Therefore, the Expanded Learning Office has
developed a rigorous schedule for the continuous professional development of staff. In
collaboration with its community-based partners, WCCUSD annually facilitates over 100
hours of professional development for direct service staff.
The Expanded Learning CQI Cycle and Activities The District supported all sites in the CQI cycle throughout the year to assess their
programs and gather data, set academic and quality improvement goals based on that
data, and make plans to achieve those goals (Figure 2).
Figure 2. CQI activities in 2017-18
!
7 Durlak, J.A., & Weissberg, R.P. (2007). The impact of after-school programs that promote personal and social skills. Chicago, IL: Collaborative for Academic Social, and Emotional Learning.
Summer 2017
•August Institute•Program profile data review
Fall 2017
•Site self-assessment•Site Visits•Quality improvement goal setting
•Quarterly reports•Ongoing professional development and technical assistance
•Facilitation of Professional Learning Communities (PLCs)
Winter 2017
•Planning with Data (PQA)
•Quality improvement goal setting
•Quarterly reports•Ongoing professional development and technical assistance
•Facilitation of Professional Learning Communities (PLCs)
Spring 2018
•Site Visits•Quarterly reports•Ongoing proffesional development and technical assistance
•Facilitation of Proffesional Learning Communities (PLCs)
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
10
Program Quality Assessment The WCCUSD Expanded Learning Office promoted a culture of observation and feedback to
support high quality practices in its Expanded Learning Programs. The Expanded Learning
Office staff conducted fall and spring site visits of each program, using the Youth and
School-Age Program Quality Assessment (PQA) tools.8
Before the WCCUSD Expanded Learning Office Staff conducted their observations, each
program conducted a self-assessment. Program staff (usually a team of leaders and
frontline staff) observed their own program and scored using the PQA rubrics. Staff then
used their notes and scores to identify strengths and areas for growth in the program to
provide specific coaching and feedback to frontline staff. The PQA scores provided a
snapshot of program quality at the time of the observation.
In addition to the self-assessment, District staff – all certified as reliable assessors by the
Weikart Center – conducted formal program observations in the fall and spring. Over time
and across individual sites in a network or school district, these observations can provide a
District-level view of program quality as experienced by youth participants.
PQA Data Findings
WCCUSD Expanded Learning Programs improved their program quality scores between fall
and spring. Elementary school programs’ overall PQA score increased from 3.40 to 3.69, a
+0.29 difference. Middle school programs’ overall PQA score increased from 2.83 to 3.03,
a +0.20 difference. In both fall and spring, elementary and middle school programs scored
greater than 4.50 in the safety domain which indicates that Expanded Learning Programs
were particularly strong when it comes to providing an emotionally and physically safe
environment to their youth. Elementary programs had great improvement in the
supportive environment domain (+0.30) and interaction domain (+0.44). Middle school
programs did not make any improvement in the interaction domain with a steady score of
2.15. Although programs scored lower in the engagement domain compared to other
domains, elementary and middle school programs increased their score from 2.52 to 2.81
(+0.29) and 1.46 to 2.08 (+0.62), respectively, demonstrating an investment in the
continuous cycle of quality improvement.9
!
8 These tools are site observation rubrics that provide reliable data on which programs are implementing high quality practices. 9 For more detailed Program Quality Scores for fall and spring by site, see Appendix B.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
11
Figure 3. Throughout the year elementary programs improved their scores across all domains
Source: District site visits, SAPQA 2017-18. Elementary school program site visits were conducted by WCCUSD Expanded Learning Office District staff with the School-Age Program Quality Assessment Tool in the fall 2017 and spring 2018, n=24.
Figure 4. Middle school programs scored low in the engagement domain, but continued to increase their score throughout the year
Source: District site visits, YPQA 2017-18. Middle school program site visits were conducted by WCCUSD Expanded Learning Office District staff with the Youth Program Quality Assessment Tool in the fall 2017 and spring 2018, n=4.
!
4.62
3.343.14
2.52
3.40
4.72
3.64 3.58
2.81
3.69
1.00
2.00
3.00
4.00
5.00
Safety Supportive Interaction Engagement Overall
Fall Spring
4.55
3.16
2.15
1.46
2.83
4.56
3.31
2.08
3.03
1.00
2.00
3.00
4.00
5.00
Safety Supportive Interaction Engagement Overall
Fall Spring
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
12
INVESTMENT IN STAFF
The District employs a youth development framework that places responsibility on its
youth workers to create an environment that is conducive to student learning and their
success. Therefore, the Expanded Learning Office makes an investment in program staff
through year-round support and professional development opportunities which ultimately
benefits students.
The Expanded Learning Office convened monthly training in the Youth Work Methods, a
series of professional development workshops on high quality staff practices for Site
Supervisors, group leaders, and certificated lead teachers. This multi-tiered professional
development opportunity builds content knowledge and leadership skills at all levels at a
program site. The Youth Work Methods are also aligned with the practices described in the
Program Quality Assessment tool used for observations.
The WCCUSD Expanded Learning Office also organized its sites into Professional Learning
Communities (PLCs). The PLCS were guided by the core values of integrity,
professionalism, and competency. Each PLC consisted of seven sites that met monthly to
analyze data, discuss challenges, share best practices, and provide peer support. The PLCs
were organized by area of interest identified by the Site Supervisors. The areas of focus
for the 2017-18 PLCs were supporting English Language Learners, developing and retaining
high quality staff, social and emotional learning, and family engagement.
Coaching and technical assistance was also provided by the District’s Expanded Learning
staff to sites throughout the year. These supports were tailored based on site needs, in
order to meet each site where they were at.
Finally, each of the three lead agencies was responsible for meeting with their program
staff prior to the start of the school year for a minimum of 20 hours. This time was used to
develop curriculum, review classroom management and pedagogical strategies, and
discuss safety protocols and program policies. Each lead agency also provided their own
professional development and coaching to their staff throughout the school year.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
13
WCCUSD EXPANDED LEARNING PARTICIPANTS YOUTH SERVED
In 2017-18, WCCUSD’s Expanded Learning Programs served 3,484 youth at 24 elementary
schools and 4 middle schools. Program participants represented 25% of all youth at the
host schools. WCCUSD served a population of students and participants in the Expanded
Learning programs that reflected diversity.
During the 2017-18 school year, most WCCUSD students resided in Richmond and San
Pablo, where more than one in five youth live in households below the federal poverty
level.10 Within the 28 school sites that hosted Expanded Learning Programs, 85% of
students were eligible for free or reduced-price meals, an indicator of family poverty.11
Figure 5. Most expanded learning participants live in Richmond and San Pablo
Source: Cityspan Attendance System for attendance records (August 21, 2017 – June 7, 2018) (N=3,484).
10 The poverty rate among children in households for whom poverty status is determined below the poverty level for the core zip codes served by WCCUSD ranges from 22%% to 33% compared to 22% for the state of California and 13% for Contra Costa County as a whole. The median income is far below the state median. In these three zip codes (94801, 94804, and 94806), median income for families ranges from $46,105 to $58,254, compared to $72,952 for the state and $98,983 for the county. Finally, while only 18% of children in the county live in households that receive public assistance, 26-41% of children in these three zip codes do so. Source: 2012-2016 American Community Survey 5-Year Estimates. 11 Source: California Department of Education DataQuest, retrieved September 2018.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
14
Notably, these youth resided in communities where more than half of the residents speak
a language other than English. Further, roughly half of those residents do not speak English
very well.12 Among the student body at these 28 school sites, 47% of youth started the
2017-18 academic year as English Language Learners.13 Similarly, within Expanded
Learning Programs, 47% of participants started the 2017-18 academic year as English
Language Learners, ranging from 14% to 73% at different sites; 12% of these participants
were reclassified to fluent English proficient by the end of the year (for more information
on reclassification rates, see page 30).14
WCCUSD Expanded Learning Programs served primarily Latino and African American
students, and at slightly different proportions than that of the regular school day. While
Latino students made up 55% of the host schools’ total student populations, they
represented 60% of youth in the expanded learning programs. Programs also served a
higher proportion of African Americans compared to the host schools (23% compared to
16%). White youth — who made up 10% of the District’s population — represented just 6%
of program participants. Similarly, Asian and Pacific Islander youth accounted for 16% of
the District’s total student count but just 10% of program participants. Programs served
roughly the same proportion of boys and girls as the District: after school youth were 49%
female and 51% male, while the District was 48% female and 52% male.
Figure 6. Programs primarily serve Latino/Hispanic and African American youth
Source: Cityspan Attendance System for attendance records (August 21, 2017 – June 7, 2018) (N=3,484).
12 Source: U.S. Census Bureau, 2012-2016 American Community Survey 5-Year Estimates 13 Source: WCCUSD demographic records (August 21, 2017 – June 7, 2018) 14 Source: WCCUSD demographic records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records
Latino/Hispanic, 60%Black/African American, 23%
Asian/Pacific Islander, 10%
White, 6% Other, 1%
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
15
ATTENDANCE
The Expanded Learning Programs run every day that school is open, 180 days a year. One
way to gauge regular attendance is to look at the average days attended per student.
Elementary youth are expected to attend five days a week. Middle school students are
expected to attend at least three days a week. Accordingly, we expect to see fewer days
attended, on average, among middle school youth (See Figure 7).
Another way is to look for the proportion of youth who attended 100 days or more, as
research suggests that youth gain additional benefits after roughly that amount of
consistent program participation.15
Figure 7. Average days attended exceeded 110 days per student
Source: Cityspan Attendance System for attendance records (August 21, 2017 – June 7, 2018) (N=3,484). For more measures of program attendance, including measures tracked to comply with grant guidelines, see Table 5 in the Appendix.
15 Huang, D., La Torre, D., Leon, S., Duong, N. and Hodson, C. (2011) “Supporting Student Success in Middle Schools: Examining the Relationship between Elementary Afterschool Program Participation and Subsequent Middle School Attainments” National Center for Research on Evaluation, Standards, and Student Testing (CRESST), University of California, Los Angeles. Retrieved from https://www.cse.ucla.edu/downloads/files/Huang.etal.AERA.paper.pdf.
120
68
ES (N=2,590) MS (N=894)
Overall Average: 112
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
16
YOUTH AND FAMILY FEEDBACK
Program Quality
Safe environment
Youth and families alike reported that their Expanded Learning
Program was a safe place, a foundational component of a high quality
program. A strong majority of students felt safe in the program (91%),
and all families agreed that the Expanded Learning Program is a safe
place for their child (100%). Slightly fewer youth reported that an
adult in the program asks for their ideas on how to solve conflicts
(78%) or that conflicts are resolved without anyone getting hurt (79%).
The staff is friendly and helpful. My daughter feels
safe and open to trying new things in the program.
- Family Member
Supportive environment
Youth and families also felt that the program was a supportive
environment with caring adult relationships. A strong majority of
youth felt there is an adult who cares about them in the program
(91%) and there is an adult who wants them to do their best (93%).
Families overwhelmingly agreed that their child feels comfortable
with the Expanded Learning Program staff (99%) and that there is an
adult in the program that their child can talk to about problems (97%).
What I like about this program is that when I need help
with something there is an adult that would be there
for me.
- Youth Participant
The staff are kind and caring. The kids respect the
adults and have a great time.
- Family Member
91% of youth
reported feeling
safe in the
program.
91% of youth felt
there is an adult
who cares about
them.
99% of families
agreed their child
feels comfortable
with the program
staff.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
17
Interactive
Youth generally reported that they have positive interactions with
their peers in the program. The majority of youth reported that they
have a friend who really cares about them (88%). Slightly fewer youth
reported that they feel like they belong in the program (80%),
however, this is an increase from last year (75% in 2016-17). Students
also reported that they get to help other students in the program
(76%). A strong majority of families agreed that their child has
opportunities to develop leadership skills in the program (97%).
One thing I like best about this program is my friends.
You always make at LEAST one new friend here. If you
are in the same grade, you get to spend a lot of time
with them and the teachers embrace it.
- Youth Participant
What I really like about after school program is that I
can learn more about how to be a better person and
because I have teachers that really care about me.
- Youth Participant
What I like best about this program is that we get to do
fun activities, we can help each other with problems,
and that there are adults who care about all of us.
- Youth Participant
What I like best in the after school program is that I
got to hang out with one of the school’s teacher and
got to do projects with my teacher like build a
community for my schools and complete projects in
independent groups. I WAS GROUP LEADER!
- Youth Participant
!
97% of families
agreed their child
has opportunities
to develop
leadership skills.
83% of youth felt
the adults in the
program listen to
what they have to
say.
88% of youth said
they have a friend
who really cares
about them.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
18
Engaging
Youth generally reported that they found the program engaging. The majority of youth
reported that the adults in the program listen to what they have to say (83%), an
increase from last year (77% in 2016-17). Less than half of youth felt they get to choose
what they do or how they do it during program activities (48%), however, this is an
improvement over last year when just 37% of youth agreed. Nearly all families said
their child has the opportunity to try new things (98%) and that the program’s activity
offerings meet their child’s interests and talents (95%).
My child has had the opportunity to extend his
instructional day in a supportive, caring environment
that emphasizes critical thinking and collaboration with
others.
- Family Member
This breadth [of activities] expands their skills,
interests, awareness of possibilities and willingness to
try something new. I think this can be very empowering
for children of all personalities. There is a diverse
group of program instructors, all who are friendly and
greet the families daily with respect. They are always
open to ideas. I really appreciate the program and all
that it offers to our daughter, and look forward to
enrolling my son next year
- Family Member
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
19
Family Engagement and Satisfaction
The majority of families reported that family engagement in the
program was welcomed. Families overwhelmingly agreed that staff
recognize them when they visit the Expanded Learning Program (99%)
and that staff listen to families’ questions or comments (99%). Slightly
fewer, though still a strong majority, of families felt that there are
opportunities for family participation in the program (94%). Beyond
engagement in the program, fewer families reported knowing more
about their child’s school day (90%) or being more involved in their
child’s school (88%).
Overall, nearly all families were satisfied or very satisfied with the
program (98%). Nearly all families reported that their child says good
things about the program (98%). Furthermore, the program serves an
important need for families; an overwhelming majority of families said
that they can go to school or work because their child is in the program
(98%).
I love the fact that the staff are very friendly and
approachable. I am able to talk to my son’s teacher
one-on-one about any concerns or questions I may have.
My son always has great things to say about the
Expanded Learning Program.
- Family Member
Student Outcomes
Social Emotional Skills
Most youth reported developing social emotional skills as a result of
their participation in the program. About three-quarters of youth felt
that the program helps them learn how to work out problems (78%),
listen to others (78%), and make friends (76%), all improvements over
last year.
I like that the adults in this program help me solve
problems and do my best at things I like to do.
- Youth Participant
98% of families
were satisfied with
the program.
78% of youth said
the program helps
them learn how to
work out problems.
99% of families felt
the program staff
listen to their
questions or
comments.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
20
Academic Behaviors
Most youth also reported developing academic skills through their
participation in the program. About three-quarters of youth said being
in the program helps them feel like a part of their school (79%), learn
things that help with school work (77%), and enjoy learning (75%).
Fewer youth felt more confident talking in class because of
participation in the program (65%). The strong majority of families
agreed that the program teaches skills that will help their child succeed
in school (97%) and that their child’s attitude towards school has
improved since coming to the program (93%).
What I like best about this program is that I get to have
fun with math and learn things that I didn't know before.
- Youth Participant
New Skills and Experiences
Most youth learned how to do something new in the program (82%),
though fewer youth learned new information about topics that interest
them (69%).
If I need help with something, I can get help from one of
the instructors. I can try out new things I haven’t tried
before.
- Youth Participant
Sense of Mastery
Most youth felt proud of something they have done in the program (85%)
and reported that participating in the program helps them work hard
towards their goals (79%). Slightly fewer felt that they are better at
something that they used to think was hard (73%).
Ever since I got motivation I started to get better and
better. I have been in the afterschool program for
more than 5 years and I'm on top of my game. I'm proud
to be in this afterschool program.
- Youth Participant
93% of families said
their child’s attitude
toward school has
improved since
coming to the
program.
82% of youth
learned how to do
something new in
the program.
85% of youth felt
proud of something
they have done in
the program.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
21
EXPANDED LEARNING PROGRAMS SUPPORT SCHOOL DAY AND DISTRICT GOALS SCHOOL DAY ALIGNMENT
Expanded learning programs can support school day success when the two are aligned.
Strong relationships and frequent communication about important topics between the
expanded learning and school day staff reinforces the efforts on both sides, ultimately
supporting positive student outcomes. In order to measure the extent to which
Expanded Learning Programs support the school day in WCCUSD, school staff completed
a survey about the perceived relationships and communication with the Expanded
Learning Program on site.
Relationships Staff were asked to rate their level of agreement with a series of statements about the
relationship between the Expanded Learning Program and the host school. Three
quarters (75%) of staff agree that there is a strong partnership between the Expanded
Learning Program and the school (Figure 8). As part of this strong partnership, roughly
three-quarters of staff agree that Expanded Learning Program staff and school day staff
are willing to collaborate (74%). Despite this willingness, fewer staff agree that the
Expanded Learning Program’s staff reach out to teachers to identify student needs
(60%) or that the program’s curriculum and instruction reinforce concepts taught during
the school day (57%).
I would like to see a stronger connection with the ELP Program and the
school day teachers. I would be happy to inform them about the concepts
we are studying in the classroom, so that they may align their program
with what is going on in the classroom during the school day. I would be
more than happy to provide supplemental materials for this as well.
- Staff
The staff work hard to align classroom curriculum to the lessons that they
teach during the Expanded Learning Program hours. Many create their own
curriculum and seek guidance from school teachers and staff in terms of
how to improve lessons and/or instruction.
- Staff
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
22
Figure 8. Staff are willing to collaborate, but are less positive about actual practice of coordination between the program and school day staff
Source: Staff survey, spring 2018 (n=475).
!
55%
57%
60%
63%
64%
68%
70%
71%
73%
74%
74%
75%
26%
15%
23%
12%
20%
7%
9%
9%
14%
5%
8%
15%
19%
27%
17%
25%
16%
25%
21%
20%
13%
21%
17%
9%
The expanded learning program has enoughcapacity to serve all interested students.
Curriculum and instruction in the expandedlearning program reinforce concepts taught during
the school day.
Expanded learning program staff reach out toteachers to identify the needs of students.
Expanded learning program staff transmitimportant information about children and parents
to appropriate school staff in a timely fashion.
School staff are encouraged to visit the program.
Expanded learning program staff follow throughwith the commitments they make to school staff.
The program is well coordinated with other afterschool activities at the school.
The Site Supervisor keeps administration informedof important decisions and issues related to
program policy.
Expanded learning program staff take care of thespace the school provides for the program.
Expanded learning program staff are responsive toideas and suggestions from school staff.
Teachers are willing to collaborate with theexpanded learning program staff.
There is a strong partnership between theexpanded learning program and the school.
Agree Disagree I don't know
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
23
Surveyed staff who visited their school’s Expanded Learning Program had very positive
views of the program.16 For example, 90% of frequent visitor staff agreed that the
Expanded Learning Program staff are responsive to ideas and suggestions from school
staff, compared to only 54% of staff who had not visited (Figure 9). Notably, a large
proportion of staff who had not visited the program responded “don’t know” for many
of the survey items.
Figure 9. Staff who have visited the program frequently are more positive than those who have never visited
Source: Staff survey, spring 2018 (n=475). Represents the proportion who responded “Agree” or “Strongly Agree”.
!
16 For full staff survey results by frequency of visits to the Expanded Learning Program, see Appendix C.
46%
43%
50%
54%
75%
82%
85%
90%
School staff are encouraged to visit the program.
Expanded learning program staff follow throughwith the commitments they make to school staff.
The Site Supervisor keeps administration informedof important decisions and issues related to
program policy.
Expanded learning program staff are responsive toideas and suggestions from school staff.
Have visited at least 5 times Have not visited
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
24
Communication Principals, Expanded Learning Lead Teachers, and Site Supervisors were also asked to
report how often administrators talk with the Site Supervisor about various topics.
Across all three roles, staff reported that administrators most frequently talk about
student discipline issues and policies (55% reported communication at least 2-3 times a
month) and the needs or progress of individual students (47% reported communication
at least 2-3 times a month; Figure 10). Staff reported that administrators communicate
less frequently about enrollment and registration levels and policies (24% reported
communication at least 2 to 3 times a month), or planning program content (20%
reported communication at least 2 to 3 times a month). Notably, school day staff
reported communicating more often with Expanded Learning Program staff more often
than in 2016-17; a larger proportion of staff indicated that they communicate at least
2-3 times a month about all the topics listed.
The Site Coordinator is extremely involved with all classes and makes sure
to communicate with teachers, administrators and ELA/ELD coach about
student needs and goals.
- Staff
I would like to see a joint effort between the school day and extended
learning. There is very little communication between regular school day
and extended learning, so we do not know what the program entails which
in turn leads us to not make a connection with the program.
- Staff
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
25
Figure 10. Staff communicate most frequently about student discipline
Source: Staff survey, spring 2018 (n=66). Only includes staff who are in a leadership position (Principal, Site Supervisor, Expanded Learning Lead Teacher) and have visited the program 5 or more times.
!
21%
9%
18%
11%
9%
9%
5%
5%
18%
30%
32%
35%
29%
20%
23%
20%
41%
36%
23%
21%
29%
32%
26%
21%
20%
24%
27%
33%
33%
38%
47%
55%
Planning program content
Enrollment/Registration levels and policies
Staffing of program
Homework assignments
Curriculum concepts being taught in school
Issues related to classroom/shared space
The needs or progress of individual students
Student discipline issues/policies
Never 1 to 2 times a semester Once a month At least 2 to 3 times a month
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
26
DISTRICT ALIGNMENT
Expanded Learning Programs in WCCUSD also support the District’s vision. In October
2017, the District presented three community campaigns to help ensure that more
WCCUSD students are prepared for their futures: Early Literacy, Reclassification, and
A-to-G Requirements. These community campaigns feed into the District’s larger
commitment to the Roadmap 2022, a five-year plan for improving student outcomes
across the District. Expanded Learning Programs in the District help to support both the
community campaigns and the Roadmap 2022 goals with their focus on improving
academic outcomes, engaging students and their families, and providing experiences to
help them prepare for their futures.
Community Campaigns
Early Literacy
Reading at grade level by 3rd grade is crucial for future academic and career success
particularly for low income students and students of color.17 Literacy development
begins well before third grade however, with early literacy experiences starting at
birth. In WCCUSD, early literacy centers around reading comprehension, fluency,
vocabulary, phonemic awareness, and writing in Pre-K through 2nd grade. Expanded
learning programs can play an important role in developing students’ literacy skills by
incorporating literacy into the programs’ existing activities in order to support the
school day reading curriculum.18
Throughout the year, several literacy assessments were administered to students.
Kindergarteners were assessed using the Kindergarten Foundational Skills assessment;
1st grade students were assessed using the STAR Early Literacy assessment; and 2nd
through 6th graders were assessed using the STAR Reading assessment.19
17 The Annie E. Casey Foundation. (2010). Early warning! Why reading by the end of third grade matters. Baltimore, MD: Fiester. Retrieved from http://www.aecf.org/m/resourcedoc/AECF-Early_Warning_Full_Report-2010.pdf 18 Rasco, C. H., Cheatham, J. B., Cheatham, S. H., & Phalen, E. M. (2012). Using Afterschool and Summer Learning to Improve Literacy Skills. In T. K. Peterson (Ed.), Expanding minds and opportunities: Leveraging the power of afterschool and summer learning for student success. Washington, DC: Collaborative Communications Group. Retrieved from http://www.expandinglearning.org/expandingminds/article/using-afterschool-and-summer-learning-improve-literacy-skills 19 For more detailed assessment data results, see Appendix D.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
27
Kindergarten Foundational Skills Assessment
Expanded Learning Program participants were more likely to meet or exceed the
standards on the Kindergarten Foundational Skills Assessment (94%) than non-program
participants (84%; Figure 11).20
Figure 11. Kindergarten Foundational Skills Assessment: Participants are more likely to meet or exceed standard
Source: Kindergarten Foundational Skills Assessment. Participants (n=52), non-participants (n=660).
Within program participants, those who attended 100 days or more of program were
slightly more likely to meet or exceed the standards (96%) than participants who
attended less than 100 days (92%; Figure 12).21
Figure 12. Kindergarten Foundational Skills Assessment: Program participants that attended 100+ were more likely to meet or exceed the standards
Source: Kindergarten Foundational Skills Assessment. Participants that attended less than 100 days (n=24), participants that attended 100 days or more (n=28).
20 Kindergarten Foundation Skills Assessment: The difference between participants and non-participants is not statistically significant. 21 Kindergarten Foundation Skills Assessment: The difference between participants who attended 100 days or more and participants who attended less than 100 days is not statistically significant.
5% 11%
6%
84%
94%
Non-participants
Participants
Needs intervention Approaching standard Meets or exceeds standard
8%
4%
92%
96%
<100 days
100+ days
Needs intervention Approaching standard Meets or exceeds standard
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
28
STAR Early Literacy Assessment
Expanded Learning Program participants’ achievement levels on the STAR Early
Literacy Assessment were similar to those of their non-participant peers (Figure 13).22
Both participants and non-participants improved their scores on the STAR Early Literacy
Assessments from fall to spring. Participants’ growth on this assessment (improvement
by 169 points) was lower than their non-participant peers (improvement by 180
points).23
Figure 13. STAR Early Literacy Assessment: Participants match their non-participant peers
Source: STAR Early Literacy Assessment Achievement Level, spring 2018. Participants (n=76), non-participants (n=170). Within program participants, participants that attended 100 days or more of program
were more likely to exceed the standards (38%) than participants who attended less
than 100 days (20%; Figure 14).24
Figure 14. STAR Early Literacy Assessment: Program participants that attended 100+ days were more likely to exceed the standards
Source: STAR Early Literacy Assessment Achievement Level, spring 2018. Participants that attended less than 100 days (n=15), participants that attended 100 days or more (n=61).!
22 STAR Early Literacy Assessment: The difference between participants and non-participants is not statistically significant. 23 STAR Early Literacy Assessment: The difference in the amount of improvement in scores from fall 2017 to spring 2018 between participants and non-participants is not statistically significant. 24 STAR Early Literacy Assessment: The difference between participants who attended 100 days or more and participants who attended less than 100 days is not statistically significant.
38%
29%
19%
26%
11%
11%
32%
34%
Non-participants
Participants
Needs intervention Approaching standard Meets standard Exceeds standard
47%
25%
27%
26%
7%
11%
20%
38%
<100 days
100+ days
Needs intervention Approaching standard Meets standard Exceeds standard
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
29
STAR Reading Assessment
Similar to the STAR Early Literacy Assessment, Expanded Learning Program
participants’ achievement levels on the STAR Reading Assessment were similar to their
non-participant peers (Figure 15).25 Both participants and non-participants improved
their scores on the Reading Assessments from fall to spring. Participants’ growth on
this assessment (improvement 69 points) was similar to their non-participant peers
(improvement by 68 points).26
Figure 15. STAR Reading Assessment: Participants match their non-participant peers
Source: STAR Reading Assessment Achievement Level, spring 2018. Participants (n=535), non-participants (n=1,031).
Further analyses showed participants who attended 100 days or more were less likely to
be classified as needing urgent intervention (49%) than their peers that attended less
than 100 days (65%; Figure 16).27
Figure 16. STAR Reading Assessment: Program participants who attended 100+ days were less likely to be classified as needing urgent intervention
Source: STAR Reading Assessment Achievement Level, spring 2018. Participants that attended less than 100 days (n=157), participants that attended 100 days or more (n=378).!
25 STAR Reading Assessment: The difference between participants and non-participants is not statistically significant. 26 STAR Reading Assessment: The difference in the amount of improvement in scores from Fall 2017 to Spring 2018 between participants and non-participants is not statistically significant. 27 STAR Reading Assessment: The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant, p<.05.
57%
54%
14%
14%
19%
22%
8%
9%
2%
1%
Non-participants
Participants
Urgent Intervention Intervention Needed On Watch Meets Benchmark Exceeds Benchmark
65%
49%
11%
15%
17%
25%
8%
9% 2%
<100 days
100+ days
Urgent Intervention Intervention Needed On Watch Meets Benchmark Exceeds Benchmark
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
30
Reclassification
English Language Learners (ELL) are a diverse, growing demographic in public schools
across the nation who may struggle academically, socially, and culturally if they are
not proficient in the English Language.28 The District has set a goal to ensure that all
ELL students reclassify from English Language Learner to Reclassified Fluent English
Proficient by 6th grade. This reclassification is pivotal in a student’s ability to achieve
academic proficiency and academic persistence; expanded learning opportunities are
primed to provide additional support to meet ELL student needs, particularly around
literacy skills.29 Analysis of reclassification rates amongst WCCUSD students indicate
that Expanded Learning participants and non-participants are reclassified as English
Proficient at similar rates (12% and 10%, respectively).30 Further, participants that
attended 100 days or more of program were more likely to be reclassified in 2017-18
(13%) than their peers that attended less than 100 days (9%).31
Table 1. Participants who attended 100+ days were more likely to be reclassified than participants who attended less than 100 days
Reclassification Rate
Participants 12%
100+ days 13%
<100 days 9%
Non-Participants 10% Source: WCCUSD demographic records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=1,625), non-participants (n=5,441), participants that attended less than 100 days (n=596), participants that attended 100 days or more (n=1,029). Percentages only include students who started the school year classified as an English Learner.
!
28 Afterschool Alliance. (2017). Afterschool providing key literacy supports to English language learner students. Retrieved from http://afterschoolalliance.org/documents/issue_briefs/issue_ELL_literacy_69.pdf 29 Ibid. 30 For more detailed English Learner and reclassification rate results, see Appendix E. 31 English Language Reclassification: The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant, p<.05.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
31
A-to-G Requirements
A-to-G requirements are a sequence of several courses that the University of California
(UC) and the California State University (CSU) systems have approved as a way to
prepare students in high school for post-secondary opportunities. Students are required
to complete these courses in order to be eligible to apply to the UC and CSU systems.
The District’s goal is to graduate students who are college-ready and prepared to
succeed in any post-secondary option they choose. Although there were no high school
Expanded Learning Programs in 2017-18, programs focused on early literacy skills,
reclassification of English Language Learners, and building math skills to successfully
complete algebra in 9th grade; foundational steps to prepare students to successfully
complete A-to-G requirements.
Expanded Learning Programs in WCCUSD supported elementary and middle school youth
with learning how to read and developing English Language skills, which may improve
high school graduation rates and completion of A-to-G courses. While elementary and
middle school programs were not explicitly tasked with college and career readiness
program offerings, several programs tied in college and career themes throughout the
year. For example, students at YMCA programs went through a 4-week curriculum
where they explored their interests and strengths and how that can be connected to a
potential career. Using that information, students researched post-secondary options
and how to achieve success in that career. Students in 5th through 8th grade researched
potential colleges and students in 7th and 8th grade learned about A-G requirements.
Taken together, Expanded Learning Programs in WCCUSD are working hard to support
the District’s three community campaigns, engaging students and their families in
promoting early literacy, reclassification, and making strides toward A-to-G course
completion.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
32
Roadmap 2022 Roadmap 2022 is the District’s five-year plan to becoming a District with achieving
students, engaged communities, and invested employees. The District envisions a
school community where 80% of its high school graduates are college ready, where
families and students report high levels of satisfaction, and where 80% of their
employees stay with the District for at least five years. The District’s Expanded
Learning Programs made progress toward these goals over the past year.
First, Expanded Learning Programs may be especially helpful in guiding WCCUSD
students on a path to achieve college readiness. In addition to focusing on early
literacy and reclassification, Expanded Learning Programs likely contribute to
strengthened youth academic outcomes, including school day attendance, grade point
average, and standardized test scores, especially for participants who attend more
than 100 days of programming.
Second, the surveys from WCCUSD Expanded Learning Programs indicated that students
and their families are highly satisfied with their programs afterschool, which may have
carry over effects to the school and the District so that families and students view the
District more positively.
Finally, the Expanded Learning Office has prioritized professional development for
expanded learning staff, which may help to reduce turn-over rates and increase the
tenure of program staff.
School Day Attendance
Expanded learning programs can build positive relationships between youth and
program staff and their peers, provide engaging enrichment activities, and provide
much needed support for struggling students, all of which give students another reason
to come to school. Youth who participated in the District’s Expanded Learning
Programs attended school at a higher rate than their non-participant peers.
Participants attended an average of 95% of school days enrolled, compared to an
average of 93% among non-participants, which translates to approximately four more
days of academic instruction.32,33
32 School day attendance: The difference between participants and non-participants is statistically significant, p<.01. 33 For more detailed school attendance and chronic absenteeism results, see Appendix F.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
33
In addition to exploring school days attended, it can also be useful to look at the
proportion of students who are frequently or chronically absent, meaning they missed
10% or more school days. Research has shown that when youth are chronically absent,
their academic achievement suffers.34 A smaller proportion of WCCUSD Expanded
Learning participants were chronically absent (18%), compared to their non-participant
peers (26%; Figure 17), aligned with the previous finding that participants attend school
at higher rates.
Figure 17. Participants attend more school than their peers and are less likely to be chronically absent
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=3,482), non-participants (n=11,499). Chronic absence defined as an absent rate of 10% or higher of the total days enrolled.
!
34 Chang, H. & Romero M. (2008). Present, engaged, and accounted for: The critical importance of addressing chronic absence in the early grades. National Center for Children in Poverty
95% 93%
Participants Non-participants
Average Attendance Rate
18%26%
Participants Non-participants
Chronic Absence Rate
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
34
Further analyses show that chronic absence rate differs between participants and non-
participants within racial/ethnic categories. In particular, Black/African American
participants have a relatively high chronic absence rate (25%; Figure 18). However, this
was significantly lower than for Black/African American non-participants (39%).35
Figure 18. Chronic absence rate between participants and non-participants differed within racial/ethnic categories
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=3,482), non-participants (n=11,499). Chronic absence defined as an absent rate of 10% or higher of the total days enrolled.
!
35 Chronic absence rate : The difference in chronic absence rate between Black/African American participants and Black/African American non-participants is statistically significant, p<.01. For more detailed chronic absenteeism results by race/ethnicity, see Appendix F.
26%
8%
25%18% 15%
28%16%
39%26%
19%
Alaska Native/American Indian
Asian/Pacific Islander
Black/African American
Latino/Hispanic
White
Participants Non-participants
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
35
Grade Point Average
GPA measures a range of academic behaviors and skills that standardized tests, focused
more exclusively on content and concepts, can miss. Therefore, GPA can better predict
student achievement, including graduating high school and enrolling and staying in
college.36 Expanded learning programs may be in a unique position to support academic
behaviors and skills such as supporting homework, promoting study habits, and
promoting collaboration and small group work. Participants achieved a slightly lower
average GPA than non-participants (2.52 compared to 2.58).37 Across all students who
had two years of GPA data, average GPA declined in 2017-18. The decline in GPA for
Expanded Learning participants was similar to their non-participants (Table 2).38
Further analyses show that participants that attended 100 days or more of program had
a significantly higher cumulative GPA in 2017-18 (2.68) than their peers that attended
less than 100 days (2.44; Table 2).39
Table 2. Participants' GPA decreased less compared to peers over two years
Grade Point Average for Middle School Students
Average Cumulative GPA in 2017-18
Average Change from 2016-17
Participants 2.52 -0.09
Non-Participants 2.58 -0.10 Source: WCCUSD GPA records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=567), non-participants (n=2,322). Students with two years of GPA data; participants (n=239) and non-participants (n=1,147).
!
36 Allensworth, E.M., Gwynne, J.A., Moore, P., and de la Torre, M. (2014). Looking Forward to High School and College: Middle Grade Indicators of Readiness in Chicago Public School. Chicago, IL: University of Chicago Consortium on Chicago School Research. 37 GPA: The difference between participants and non-participants is not statistically significant. 38 GPA: The difference in GPA change between 2016-17 and 2017-18 between participants and non-participants is not statistically significant. 39 GPA: The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant, p<.01.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
36
English Language Arts and Math Assessments
Students in 3rd grade and up take the Smarter Balanced Assessment Consortium (SBAC),
a computer-based assessment that measures content knowledge linked to the Common
Core State Standards in English Language Arts (ELA) and Mathematics. The test provides
an achievement level (not meeting, nearly meeting, meeting, or exceeding the
standard), based on the scale score (overall numerical score based on the total number
of correct answers) and the student’s grade level. Designed primarily for school
accountability, the tests can be an indicator of relative success among sub groups
within a school or district, such as the relative academic performance of expanded
learning participants and their non-participant peers.
Expanded Learning participants’ achievement levels follow the pattern of their non-
participant peers in both the SBAC ELA and math assessment. Overall, the majority of
all students did not meet ELA or math standards. The proportion of WCCUSD students
(both participants and non-participants) who did not meet the ELA or Math standard
was considerably higher than the state, where 28% of students did not meet the ELA
standard and 35% did not meet the math standard (Figure 19).
Figure 19. Participant achievement levels follow the peers’ patterns
! Standard Not Met ! Standard Nearly Met ! Standard Met ! Standard Exceeded
Source: WCCUSD SBAC records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants with ELA Scores (n=2,238), non-participants with ELA Scores (n=6,443), participants with math scores (n=2,270), and non-participants with math scores (n=6,521); Statewide comparisons drawn from the 2018 SBAC Results, includes scores from students in 3rd-11th grade, retrieved from https://caaspp.cde.ca.gov/sb2018/.
23%
22%
22%
28%
49%
56%
29%
20%
15%
21%
9%
7%
State
Non-Participants
Participants
ELA
26%
25%
24%
35%
56%
60%
20%
11%
12%
19%
7%
5%
State
Non-Participants
Participants
Math
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
37
Further analyses showed that participants that attended 100 days or more of program
scored better on both the SBAC ELA and math assessment than those that attended less
than 100 days.40,41 About a quarter of participants who attended more than 100 days
met or exceeded the ELA standard (24%), compared to 19% of their peers who attended
less than 100 days. Scores on the SBAC Math followed the same pattern: 17% of
participants that attended 100 days or more met or exceeded the Math standard,
compared to 14% of their peers that attended less than 100 days. In particular, more
Asian/Pacific Islander, Black/African American, and Latino/Hispanic students in grades
4-6 that attended 100 days or more met or exceeded the ELA and Math standards than
their peers from the same ethnic category that attended less than 100 days.42
!
40 SBAC ELA: The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant, p<.01. 41 SBAC Math: The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant, p<.01. 42 For more detailed SBAC assessment data results by ethnic category, see Appendix D.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
38
RECOMMENDATIONS
Based on the data collected in 2017-18 and the District-wide goals set forth in its Community
Campaigns and Roadmap 2022, we present the following recommendations:
In 2017-18, Expanded Learning Programs at elementary and middle schools ensured a
safe environment for participating youth. Continue to reiterate strong practices in the
safe and supportive environment domains, as they are the foundation to high-quality
programs. To continue to improve PQA scores, prioritize and emphasize strong
practices in the interaction and engagement domains. In particular, elementary school
programs should continue to encourage participants to practice school-age leadership
skills; middle school programs should focus their practice on providing opportunities for
youth to act as group facilitators or mentors to other students and partnering with
adults. All sites should include more intentional planning, choice, and reflection
practices.
Eight in 10 youth (82%) reported that they learned how to do something new in their
Expanded Learning Program and nearly 7 in 10 (69%) reported they learned information
about a new topic that interested them. Related to emphasizing practices in the
interaction and engagement PQA domains, programs can deepen their students’
engagement by creating opportunities for all participants to have new experiences and
make authentic and meaningful choices that will allow them to better connect with the
program content. One way to increase the proportion of youth who say they learned
information about a topic that interested them is to solicit input from youth on topics
that interest them and design program activities that relate to those topics. Youth may
not realize that they are learning new skills and information, even when this is the
case. The District and Agency Directors, along with Site Supervisors should encourage
program line staff to lead participants in reflection after activities about whether they
just learned something new.
Continue to focus on sustaining or improving PQA scores in all four domains of quality.
Cultivate authentic youth voice and choice experiences to broaden horizons and deepen program engagement.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
39
Participants who attended 100 days or more of programming experienced greater
benefits than participants who attended less than 100 days, including:
•! higher reclassification rates for ELL participants;
•! higher GPA;
•! higher school day attendance and lower chronic absence rates;
•! and they were more likely to meet or exceed standards across standardized
assessments.
The average elementary school participant attended 120 days of programming,
however, the average middle school participant attended only 68 days. While program
models and attendance expectations differ between school levels, the demonstrated
value of attending program more frequently is clear. Middle school programs should
develop strategies to engage students and increase their retention rates with the goal
of increasing the average number of days attended.
Relative to the other items on the staff survey, some of the lower rated items on the
staff survey included:
•! the curriculum and instruction in the Expanded Learning Programs reinforce
concepts taught during the school day;
•! program staff reach out to teachers to identify needs of specific students;
•! and that program staff communicate importation information about children
and parents to the appropriate school days staff in a timely manner.
Meanwhile, about 1 in 5 surveyed leadership staff reported that they never
communicate about planning program content. However, surveyed staff who frequently
visited the program held much more positive views of the program compared to those
who had not visited the program. Increasing the program’s visibility with the school day
staff and broader school community can help make others more aware of what the
program does. The more staff and community members come into contact with the
program, the more likely they are to understand its benefits, including the ways in
which it is aligned with the school day. Program, school, and District leadership,
alongside any identified expanded learning champions on site, should focus on
encouraging school day staff to visit the program.
Encourage higher retention rates, especially for middle school participants.
Increase the visibility of Expanded Learning Programs on school campuses to better align with the school day.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
40
APPENDICES APPENDIX A. PROGRAM SITES, LEAD AGENCIES, AND COLLABORATIVE PARTNERS
The District partnered with three community-based lead agencies to run 28 Expanded Learning
Programs for the 2016-17 school year: Bay Area Community Resources, Love. Learn. Success., and
YMCA of the East Bay (Table 3). Across these three agencies, Expanded Learning Programs were funded
through a mix of state and federal funds. All sites, regardless of lead agency, were open 5 days per
week after school until 6:00pm and offered their participants academic support, enrichment, and
recreational activities on-site. Each site was run by a Site Supervisor who managed a set of group
leaders employed by the lead agency at the site. Each site was also supported by the District’s
Expanded Learning Office, its lead agency’s program manager, and school staff (especially the principal
and designated lead teacher).
Table 3. Expanded Learning Sites and Lead Agencies Bay Area Community Resources Love. Learn. Success. YMCA of the East Bay
Bayview Lincoln Fairmont Coronado
Chavez Mira Vista Ford Crespi
DeJean Nystrom Grant Downer
Dover Riverside King Murphy
Korematsu Stege Lake Peres
Helms Washington Montalvin Sheldon
Highland Wilson Verde Tara Hills
Additionally, the District facilitated quarterly collaborative meetings with community partners in an
effort to inform and coordinate services across the 28 Expanded Learning. These quarterly meetings
provided the opportunity for stakeholders to share information with one another, increase alignment
between the District and its partners, and continue with efforts to support a cycle of continuous
quality improvement.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
41
Table 4. Enrollment, Attendance, and Retention by Program
Lead Agency /Program
Total Youth Served
Average Daily Attendance Youth Participation
Goal Actual % Progress
Toward Annual Goal*
Average Days Attended
Average Attendance
Rate
Average Enrollment
Rate
Bay Area Community Resources
Bayview 108 83 69 83% 116 64% 87%
Chavez 104 83 73 87% 126 70% 86%
Dover 167 126 99 78% 106 59% 87%
Highland 100 86 73 85% 132 73% 89%
Lincoln 111 83 75 90% 121 67% 93%
Mira Vista 110 83 79 95% 129 72% 91%
Nystrom 117 83 83 100% 128 71% 89%
Riverside 87 83 65 77% 133 74% 91%
Stege 104 83 65 78% 113 63% 87%
Washington 107 83 70 84% 118 65% 84%
Wilson 108 83 74 89% 123 68% 94%
Love. Learn. Success
Fairmont 113 83 82 98% 130 72% 89%
Ford 163 141 117 83% 129 72% 86%
Grant 204 161 110 68% 97 54% 85%
King 112 72 55 76% 88 49% 84%
Lake 182 137 123 90% 122 68% 89%
Montalvin 113 83 72 86% 114 64% 90%
Verde 102 74 58 78% 102 57% 84%
YMCA of the East Bay
Coronado 164 127 96 75% 105 58% 79%
Downer 139 111 94 85% 121 67% 86%
Murphy 103 83 78 94% 137 76% 86%
Peres 96 83 78 94% 146 81% 92%
Sheldon 106 83 76 91% 129 72% 90%
Tara Hills 109 83 81 98% 135 75% 85%
ES Total 118 95 81 85% 120 66% 87%
Bay Area Community Resources
DeJean 105 49 37 74% 63 35% 79%
Helms 243 111 90 81% 67 37% 65%
Korematsu 133 86 53 62% 72 40% 77%
YMCA of the East Bay
Crespi 97 51 36 71% 67 37% 72%
MS Total 145 74 54 73% 68 38% 72%
Overall/Average 125 92 77 84% 112 62% 85%
Source: Cityspan Attendance System for attendance records from 8/21/17 through 6/7/18 (N=3,484). * Shading indicates sites that did not meet the 85% threshold aligned with CDE attendance goals. Average Attendance Rate = Average Days Attended / 180 Days Average Enrollment Rate = Average Days Attended / Average Days between First Day Attended and Last Day Attended
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
42
APPENDIX B. SITE VISIT METHODOLOGY AND THE PROGRAM QUALITY ASSESSMENT TOOL
Site visits provide insight into point of service components that drive program quality. The District
continued to use the Program Quality Assessment Tool (PQA) to define and measure quality in its
Expanded Learning Programs. The PQA includes four domains of program quality:
•! Safe Environment – Youth experience both physical and emotional safety. The program
environment is safe and sanitary. The social environment is safe.
•! Supportive Environment – Adults support youth to learn and grow. Adults support youth with
opportunities for active learning, for skill building, and to develop healthy relationships.
•! Interaction – There is a positive peer culture in the program, encouraged and supported by
adults. Youth support each other. Youth experience a sense of belonging. Youth participate in
small groups as members and as leaders. Youth have opportunities to partner with adults.
•! Engagement – Youth experience positive challenges and pursue learning. Youth have
opportunities to plan, make choices, and learn from their experiences.
These domains are inter-related and build upon one another. Broadly speaking, programs need to
assure that youth enjoy a Safe and Supportive Environment before working to establish high quality
Interaction and Engagement. For example, a program in which young people are afraid to try new
things for fear of being ridiculed by others - an example of an unsupportive environment - is not likely
to be an interactive, engaging place for children.
Research indicates that the foundational programmatic elements of physical and emotional safety
(described in the Safe and the Supportive Environment domains) support high quality practices in other
domains. In general, programs’ ratings will be higher for the foundational domains than for Interaction
or Engagement. The following page visualizes the relationship between each of the PQA quality
domains.!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
43
Figure 20. Program Quality Assessment Domains
Source: Adapted from Youth PQA Handbook by High/Scope Educational Research Foundation, 2007.
All observations were conducted using the School-Age Program Quality Assessment (SAPQA) or the
Youth Program Quality Assessment (YPQA), research-based point of service quality observation tools
used by out-of-school time programs nationally. The tool uses a rating system where a score of:
•! 1 indicates that the practice was not observed or did not meet minimum
standards;
•! 3 indicates a basic standard of practice;
•! 5 indicates a strong practice.
WCCUSD Expanded Learning Program Office staff have been certified as statistically reliable raters by
the Weikart Center for Youth Program Quality. Note: we report the fall, spring, and average data from
the District’s site visits in this report.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
44
Table 5. Fall PQA Scores by Site Level, Grade Level, and Overall Lead Agency/
Program Safe Environment
Supportive Environment
Interaction Engagement Overall PQA Score
Bay Area Community Resources
Bayview 5.00 2.77 2.83 3.58 3.55
Cesar E. Chavez 4.68 3.56 2.83 2.50 3.39
Dover 4.68 4.09 4.22 3.50 4.12
Highland 4.84 3.87 4.00 3.00 3.93
Lincoln 4.84 4.23 3.89 3.33 4.07
Mira Vista 4.84 3.27 2.67 2.17 3.24
Nystrom 4.68 3.39 3.50 3.00 3.64
Riverside 4.00 2.85 3.00 2.08 2.98
Stege 4.16 2.85 2.83 2.33 3.05
Washington 4.32 3.51 3.08 2.75 3.42
Wilson 4.68 3.09 3.11 2.50 3.35
Love. Learn. Success.
Grant 4.84 3.53 3.33 2.67 3.59
Fairmont 4.92 2.71 3.39 3.25 3.57
Ford 4.80 3.45 3.39 2.00 3.41
King 4.48 3.99 2.94 2.33 3.44
Lake 4.68 2.89 3.28 2.50 3.34
Montalvin 4.48 3.09 2.25 2.00 2.96
Verde 4.84 3.59 2.83 2.50 3.44
YMCA of the East Bay
Coronado 4.16 2.21 2.92 2.17 2.86
Downer 4.84 3.83 3.28 2.33 3.57
Murphy 4.76 3.00 2.72 2.25 3.18
Peres 4.60 4.07 3.13 3.25 3.76
Sheldon 4.52 3.51 3.58 1.50 3.28
Tara Hills 4.20 2.75 2.29 1.00 2.56
ES Average 4.62 3.34 3.14 2.52 3.40
Bay Area Community Resources
DeJean 4.52 2.67 2.42 1.33 2.73
Helms 4.36 3.49 1.75 1.50 2.77
Korematsu 4.56 3.23 2.42 1.67 2.97
YMCA of the East Bay
Crespi 4.76 3.25 2.00 1.33 2.84
MS Average 4.55 3.16 2.15 1.46 2.83
Overall Average 4.61 3.31 3.00 2.37 3.32
Source: District site visits, SAPQA/YPQA scores, fall 2017 (N=28).
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
45
Table 6. Spring PQA Scores by Site Level, Grade Level, and Overall Lead Agency/
Program Safe Environment
Supportive Environment
Interaction Engagement Overall PQA Score
Bay Area Community Resources
Bayview 4.80 3.69 4.00 2.50 3.75
Cesar E. Chavez 4.84 3.60 3.94 3.17 3.89
Dover 4.84 4.45 4.44 4.00 4.43
Highland 4.76 3.67 4.00 3.00 3.86
Lincoln 4.64 4.25 4.06 3.33 4.07
Mira Vista 4.87 3.69 3.67 3.50 3.93
Nystrom 4.64 3.64 3.67 2.50 3.61
Riverside 4.52 3.17 2.78 3.08 3.39
Stege 4.84 3.80 3.00 2.00 3.41
Washington 4.80 3.23 3.33 3.67 3.76
Wilson 4.64 2.85 3.33 1.75 3.14
Love. Learn. Success.
Grant 4.84 3.39 3.11 2.00 3.33
Fairmont 4.67 3.51 3.33 2.33 3.46
Ford 4.74 3.56 3.33 2.33 3.49
King 4.64 3.37 3.33 2.50 3.46
Lake 4.48 3.28 3.33 1.92 3.25
Montalvin 4.34 2.77 3.11 2.25 3.12
Verde 4.84 3.26 2.89 2.58 3.39
YMCA of the East Bay
Coronado 4.72 4.13 3.83 3.67 4.09
Downer 4.84 4.09 4.00 2.75 3.92
Murphy 4.42 3.11 3.67 2.33 3.38
Peres 5.00 4.17 3.83 3.50 4.13
Sheldon 5.00 4.72 4.44 3.67 4.46
Tara Hills 4.60 3.96 3.54 3.08 3.80
ES Average 4.72 3.64 3.58 2.81 3.69
Bay Area Community Resources
DeJean 4.56 3.53 2.29 2.00 3.10
Helms 4.76 3.57 2.58 2.00 3.23
Korematsu 4.76 3.31 1.71 2.33 3.03
YMCA of the East Bay
Crespi 4.16 2.84 2.00 2.00 2.75
MS Average 4.56 3.31 2.15 2.08 3.03
Overall Average 4.70 3.59 3.38 2.71 3.59
Source: District site visit SAPQA/YPQA scores, spring 2018 (N=28).
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
46
Table 7. Fall and Spring PQA Scores by Site Level, Grade Level, and Overall Lead Agency/
Program Safe Environment
Supportive Environment
Interaction Engagement Overall PQA Score
Bay Area Community Resources
Bayview 4.90 3.23 3.42 3.04 3.65
Cesar E. Chavez 4.76 3.58 3.39 2.83 3.64
Dover 4.76 4.27 4.33 3.75 4.28
Highland 4.80 3.77 4.00 3.00 3.89
Lincoln 4.74 4.24 3.97 3.33 4.07
Mira Vista 4.85 3.48 3.17 2.83 3.58
Nystrom 4.66 3.51 3.58 2.75 3.63
Riverside 4.26 3.01 2.89 2.58 3.19
Stege 4.50 3.33 2.92 2.17 3.23
Washington 4.56 3.37 3.21 3.21 3.59
Wilson 4.66 2.97 3.22 2.13 3.25
Love. Learn. Success.
Grant 4.84 3.46 3.22 2.33 3.46
Fairmont 4.79 3.11 3.36 2.79 3.51
Ford 4.77 3.50 3.36 2.17 3.45
King 4.56 3.68 3.14 2.42 3.45
Lake 4.58 3.09 3.31 2.21 3.30
Montalvin 4.41 2.93 2.68 2.13 3.04
Verde 4.84 3.42 2.86 2.54 3.42
YMCA of the East Bay
Coronado 4.44 3.17 3.38 2.92 3.48
Downer 4.84 3.96 3.64 2.54 3.75
Murphy 4.59 3.05 3.19 2.29 3.28
Peres 4.80 4.12 3.48 3.38 3.94
Sheldon 4.76 4.11 4.01 2.58 3.87
Tara Hills 4.40 3.35 2.92 2.04 3.18
ES Average 4.67 3.49 3.36 2.66 3.55
Bay Area Community Resources
DeJean 4.54 3.10 2.35 1.67 2.92
Helms 4.56 3.53 2.17 1.75 3.00
Korematsu 4.66 3.27 2.06 2.00 3.00
YMCA of the East Bay
Crespi 4.46 3.05 2.00 1.67 2.79
MS Average 4.56 3.23 2.15 1.77 2.93
Overall Average 4.65 3.45 3.19 2.54 3.46
Source: District site visits, SAPQA/YPQA scores, average of fall 2017 and spring 2018 (N=28).!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
47
APPENDIX C. YOUTH, FAMILY, AND STAFF SURVEYS
Survey Methodology Surveys for participating youth, families, and staff were fielded between February 2018 and March
2018, and distributed with the support of Site Supervisors. Surveys for all youth were online. Surveys
for families at elementary schools were online, and surveys for families at middle schools were
provided on paper. Both the youth and family surveys were available in English and Spanish; youth and
families self-selected the language in which they preferred to complete the survey. Staff surveys were
available in English only.
In this evaluation, the youth survey results are used to gather participants’ perspective on the quality
and experience in WCCUSD’s Expanded Learning Programs.
Similarly, family survey results are used in the evaluation to understand the family perspective on the
quality of the Expanded Learning Program their child(ren) attended, gather information regarding the
growth they see in the specified outcome areas described in the report, and to learn about families’
satisfaction with the Expanded Learning Program.
Survey Response Rates
Youth Survey
Site Supervisors were tasked with administering the youth survey to as many program participants as
possible, in grades 3 and higher (in consideration of reading ability). The total survey count (n=1,590)
represents approximately 74% of the 2,159 youth who attended WCCUSD’s Expanded Learning Programs
on the average day, and approximately 46% of the 3,484 total youth served (unduplicated) by the
District’s Expanded Learning Programs during the 2017-18 program year.
Family Survey
Site Supervisors were also asked to administer the family survey to as many families or caregivers of
program participants as possible. Families were asked to complete only one survey, even if they had
multiple children in the program, and they were asked to think about their oldest child when
completing the survey. Unlike the youth survey, families that represented all grades (K-8) were
encouraged to respond. The total family survey count (n=1,447) represents approximately 42% of the
3,484 total youth served (unduplicated) by the District’s Expanded Learning programs during the 2017-
18 program year.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
48
Staff Survey
Program staff also distributed the staff survey to staff members at their schools who have interacted
with the Expanded Learning Program, including the school’s principal, school day teachers, the after
school program lead teacher, school counselors, the Site Supervisors themselves, and any others they
felt could offer their perspective on their program. A total of 475 staff members completed a valid
survey, which is an average of 20 staff members per site.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
49
Table 8. Youth Survey: Program Quality Results
Program I feel safe in this program.
If I have a conflict, the adults in this program ask for my ideas about how to
resolve it.
In this program, I
resolve conflicts without anyone
getting hurt.*
There is an adult who
cares about me in this program.
In this program,
there is an adult who
wants me to do my best.
I feel like I belong at this
program.
I get to help other
students in this program.
I have a friend in this program who really cares about me.
The adults in this program listen to what I have to say.
During program
activities, I get to choose what to do or how to do it.
Bayview (n=55) 96% 71% 90% 95% 93% 89% 71% 89% 85% 45%
Chavez (n=40) 100% 78% 100% 98% 100% 90% 83% 95% 93% 78%
Coronado (n=64) 83% 75% 71% 92% 94% 67% 82% 89% 85% 41%
Dover (n=68) 85% 58% 89% 88% 86% 75% 78% 88% 72% 46%
Downer (n=71) 97% 85% 94% 94% 97% 89% 83% 93% 93% 46%
Fairmont (n=51) 96% 86% 81% 84% 94% 82% 69% 80% 80% 37%
Ford (n=79) 66% 67% 41% 81% 77% 53% 66% 79% 62% 31%
Grant (n=91) 97% 84% 91% 86% 99% 93% 76% 91% 92% 46%
Highland (n=44) 95% 66% 100% 100% 93% 82% 75% 80% 82% 48%
King (n=33) 82% 76% 73% 85% 91% 70% 53% 88% 73% 44%
Lake (n=91) 91% 78% 76% 91% 92% 77% 75% 88% 79% 45%
Lincoln (n=54) 91% 76% 75% 94% 96% 63% 76% 85% 70% 50%
Mira Vista (n=52) 92% 73% 87% 100% 100% 75% 84% 88% 76% 35%
Montalvin (n=49) 96% 82% 69% 88% 96% 80% 65% 82% 84% 55%
Murphy (n=55) 95% 80% 72% 91% 93% 87% 65% 80% 87% 46%
Nystrom (n=57) 93% 62% 78% 93% 93% 84% 79% 88% 89% 45%
Peres (n=49) 88% 86% 76% 96% 98% 84% 94% 96% 92% 59%
Riverside (n=49) 90% 78% 69% 90% 90% 82% 82% 92% 81% 45%
Sheldon (n=63) 98% 92% 88% 95% 98% 92% 90% 95% 92% 49%
Stege (n=44) 95% 79% 89% 89% 91% 80% 77% 88% 74% 24%
Tara Hills (n=66) 95% 80% 81% 94% 97% 81% 78% 92% 86% 55%
Verde (n=32) 94% 94% 90% 88% 97% 84% 91% 91% 94% 56%
Washington (n=57) 89% 84% 92% 88% 89% 72% 79% 84% 81% 47%
Wilson (n=56) 96% 96% 94% 96% 100% 95% 91% 93% 93% 63%
ES Total 91% 78% 80% 91% 94% 80% 78% 88% 83% 47%
Crespi (n=38) 66% 66% 58% 74% 79% 61% 37% 78% 63% 18%
DeJean (n=34) 91% 82% 73% 88% 94% 76% 68% 91% 88% 53%
Helms (n=94) 97% 79% 84% 96% 95% 90% 76% 90% 93% 76%
Korematsu (n=54) 91% 64% 77% 87% 83% 63% 63% 83% 66% 36%
MS Total 89% 74% 76% 89% 89% 76% 65% 87% 80% 53%
Total 91% 78% 79% 91% 93% 80% 76% 88% 83% 48%
Source: Youth Survey, Feb-March 2018 (N=1,590). Represents the proportion who responded positively. *This item was asked only of youth in 5th grade and up.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
50
Table 9. Youth Survey: Direct Outcomes Results
Program This program helps me feel like a part of my
school.
I learn how to do things in this program that help
with my school work.
This program helps me enjoy learning.*
Because of this program, I feel more confident
talking in class.
This program helps me listen to others.
In this program, I learn how to work out my
problems.
Bayview (n=55) 84% 78% 76% 76% 84% 80%
Chavez (n=40) 100% 100% 93% 100% 95% 98%
Coronado (n=64) 81% 79% 71% 61% 79% 84%
Dover (n=68) 81% 80% 70% 58% 71% 70%
Downer (n=71) 94% 92% 92% 70% 90% 97%
Fairmont (n=51) 84% 62% 70% 67% 85% 77%
Ford (n=79) 58% 50% 55% 42% 48% 59%
Grant (n=91) 91% 89% 92% 79% 89% 87%
Highland (n=44) 77% 84% 82% 100% 80% 84%
King (n=33) 75% 64% 79% 73% 73% 67%
Lake (n=91) 81% 72% 72% 61% 77% 79%
Lincoln (n=54) 72% 74% 64% 65% 74% 72%
Mira Vista (n=52) 82% 71% 79% 80% 87% 83%
Montalvin (n=49) 78% 79% 83% 53% 81% 77%
Murphy (n=55) 85% 73% 71% 56% 81% 78%
Nystrom (n=57) 82% 80% 85% 59% 85% 79%
Peres (n=49) 73% 71% 78% 50% 90% 82%
Riverside (n=49) 78% 79% 84% 50% 82% 82%
Sheldon (n=63) 89% 82% 80% 77% 80% 85%
Stege (n=44) 84% 90% 76% 50% 75% 81%
Tara Hills (n=66) 79% 82% 73% 69% 80% 83%
Verde (n=32) 88% 90% 94% 80% 90% 87%
Washington (n=57) 67% 69% 73% 60% 72% 76%
Wilson (n=56) 93% 87% 94% 82% 89% 87%
ES Total 81% 78% 78% 65% 80% 80%
Crespi (n=38) 37% 37% 29% 42% 47% 37%
DeJean (n=34) 74% 91% 62% 73% 85% 74%
Helms (n=94) 82% 85% 72% 78% 76% 77%
Korematsu (n=54) 42% 50% 40% 47% 37% 48%
MS Total 63% 69% 55% 63% 63% 62%
Total 79% 77% 75% 65% 78% 78%
Source: Youth Survey, Feb-March 2018 (N=1,590). Represents the proportion who responded positively. *This item was asked only of youth in 5th grade and up.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
51
Table 10. Youth Survey: Direct Outcomes Results (Continued)
Program This program helps me
to make friends.
In this program, I learned how to do something new.*
In this program, I learned new information
about a topic that interests me.
I am better at something I used to think was hard.
This program helps me work hard toward my
goals.
I feel proud of things I’ve done in this program.
Bayview (n=55) 80% 82% 81% 71% 80% 91%
Chavez (n=40) 95% 100% 100% 95% 95% 95%
Coronado (n=64) 73% 84% 65% 76% 81% 81%
Dover (n=68) 73% 79% 74% 68% 77% 76%
Downer (n=71) 93% 93% 87% 82% 96% 94%
Fairmont (n=51) 89% 87% 81% 66% 77% 89%
Ford (n=79) 56% 51% 45% 51% 57% 69%
Grant (n=91) 79% 92% 94% 84% 90% 98%
Highland (n=44) 81% 86% 100% 79% 86% 93%
King (n=33) 70% 75% 73% 63% 72% 94%
Lake (n=91) 80% 76% 62% 77% 78% 82%
Lincoln (n=54) 60% 81% 68% 68% 74% 79%
Mira Vista (n=52) 77% 83% 67% 71% 84% 94%
Montalvin (n=49) 85% 85% 60% 77% 85% 89%
Murphy (n=55) 76% 82% 52% 63% 78% 83%
Nystrom (n=57) 73% 80% 77% 69% 89% 91%
Peres (n=49) 77% 92% 63% 71% 78% 92%
Riverside (n=49) 73% 88% 67% 67% 82% 83%
Sheldon (n=63) 79% 89% 73% 77% 80% 98%
Stege (n=44) 71% 83% 76% 90% 83% 93%
Tara Hills (n=66) 75% 89% 69% 82% 85% 82%
Verde (n=32) 81% 100% 70% 84% 90% 94%
Washington (n=57) 62% 80% 72% 65% 73% 73%
Wilson (n=56) 93% 83% 76% 87% 87% 94%
ES Total 77% 83% 71% 74% 81% 87%
Crespi (n=38) 51% 43% 47% 37% 37% 50%
DeJean (n=34) 76% 97% 76% 71% 76% 82%
Helms (n=94) 84% 85% 76% 79% 84% 89%
Korematsu (n=54) 70% 57% 45% 52% 46% 56%
MS Total 74% 73% 64% 64% 65% 73%
Total 76% 82% 69% 73% 79% 85%
Source: Youth Survey, Feb-March 2018 (N=1,590). Represents the proportion who responded positively. *This item was asked only of youth in 5th grade and up.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
52
Table 11. Family Survey: Direct Outcomes and Academic Behavior Results
Program This ELP is a safe
place for my child.
My child feels comfortable with
the ELP staff.
There is an adult in this program
my child can talk to about her or his problems.
In this program, my child has
opportunities to try new things.
This program provides activities
that meet my child’s interests
and talents.
In this program, my child has
opportunities to develop
leadership skills.
This program teaches skills that help my child in
school.
My child's attitude toward school has
improved since coming to this
program.
Bayview (n=49) 100% 98% 94% 98% 100% 100% 100% 96%
Chavez (n=94) 100% 100% 100% 100% 91% 98% 97% 97%
Coronado (n=66) 100% 97% 97% 100% 97% 98% 95% 88%
Dover (n=48) 100% 96% 95% 100% 100% 95% 98% 98%
Downer (n=57) 100% 100% 96% 98% 100% 100% 100% 91%
Highland (n=50) 100% 100% 98% 100% 98% 98% 100% 98%
Fairmont (n=31) 97% 90% 96% 94% 93% 96% 97% 90%
Ford (n=41) 100% 100% 95% 100% 100% 97% 100% 100%
Grant (n=78) 100% 99% 94% 100% 97% 96% 97% 96%
King (n=18) 100% 94% 100% 100% 100% 100% 94% 100%
Lake (n=45) 100% 100% 95% 100% 100% 98% 98% 98%
Lincoln (n=64) 100% 100% 98% 100% 98% 98% 98% 100%
Mira Vista (n=55) 100% 100% 100% 94% 89% 94% 98% 90%
Montalvin (n=51) 96% 96% 92% 98% 96% 96% 94% 90%
Murphy (n=57) 100% 100% 98% 98% 98% 96% 98% 96%
Nystrom (n=67) 100% 98% 98% 100% 98% 100% 100% 97%
Peres (n=48) 100% 98% 93% 98% 96% 98% 96% 89%
Riverside (n=53) 100% 100% 100% 100% 98% 100% 100% 98%
Sheldon (n=62) 97% 97% 97% 97% 93% 97% 94% 95%
Stege (n=63) 100% 98% 98% 95% 83% 97% 84% 81%
Tara Hills (n=53) 100% 100% 100% 100% 98% 100% 98% 98%
Verde (n=38) 100% 100% 100% 100% 100% 97% 100% 100%
Washington (n=64) 98% 97% 93% 91% 73% 85% 90% 82%
Wilson (n=54) 100% 100% 100% 96% 94% 100% 98% 91%
ES Total 100% 99% 97% 98% 95% 97% 97% 94%
Crespi (n=22) 100% 95% 89% 90% 76% 83% 83% 68%
DeJean (n=34) 100% 97% 96% 97% 91% 89% 97% 86%
Helms (n=42) 100% 100% 97% 95% 95% 92% 98% 83%
Korematsu (n=43) 100% 100% 97% 100% 97% 94% 97% 89%
MS Total 100% 99% 96% 96% 92% 91% 95% 83%
Total 100% 99% 97% 98% 95% 97% 97% 93%
Source: Family Survey, Feb-March 2018 (N=1,447). Represents the proportion who responded “Agree” or “Strongly Agree”. The response option “I don’t know” was excluded from analysis. Actual survey items said “expanded learning program” instead of “ELP”.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
53
Table 12. Family Survey: Engagement and Satisfaction Results
Program At least one ELP staff recognizes me when I visit.
The ELP staff members listen to me when I have a
question or comment.
There are opportunities for
family participation in this program.
Because my child is in this ELP, I am more involved at my child’s school.
Because my child is in this ELP, I
know more about what goes on in the school day.
My child tells me good things about
this ELP.
Because my child is in this program, I can go to school
or work.
How satisfied are you with this
expanded learning program?
Bayview (n=49) 98% 98% 100% 98% 98% 98% 100% 96%
Chavez (n=94) 100% 100% 97% 89% 87% 99% 100% 100%
Coronado (n=66) 97% 97% 96% 84% 85% 97% 100% 95%
Dover (n=48) 96% 98% 90% 90% 93% 98% 98% 98%
Downer (n=57) 100% 100% 96% 80% 86% 96% 96% 100%
Highland (n=50) 100% 100% 98% 94% 94% 100% 96% 100%
Fairmont (n=31) 97% 90% 93% 87% 86% 86% 90% 90%
Ford (n=41) 98% 100% 94% 89% 95% 100% 100% 97%
Grant (n=78) 97% 97% 95% 97% 96% 97% 100% 99%
King (n=18) 100% 100% 100% 88% 93% 100% 100% 100%
Lake (n=45) 98% 100% 90% 91% 93% 100% 95% 100%
Lincoln (n=64) 100% 100% 98% 100% 98% 98% 100% 100%
Mira Vista (n=55) 100% 100% 93% 77% 87% 96% 100% 100%
Montalvin (n=51) 96% 98% 94% 88% 88% 96% 96% 100%
Murphy (n=57) 100% 100% 98% 93% 91% 100% 98% 100%
Nystrom (n=67) 100% 100% 97% 95% 97% 100% 100% 98%
Peres (n=48) 98% 98% 98% 91% 91% 98% 100% 98%
Riverside (n=53) 100% 100% 98% 96% 92% 100% 98% 100%
Sheldon (n=62) 97% 97% 95% 90% 95% 97% 97% 97%
Stege (n=63) 100% 100% 73% 80% 76% 95% 100% 100%
Tara Hills (n=53) 100% 100% 100% 81% 90% 100% 100% 100%
Verde (n=38) 100% 97% 97% 91% 100% 100% 97% 97%
Washington (n=64) 100% 100% 87% 71% 74% 95% 100% 94%
Wilson (n=54) 98% 100% 98% 85% 89% 100% 100% 98%
ES Total 99% 99% 95% 89% 90% 98% 99% 98%
Crespi (n=22) 100% 100% 88% 71% 76% 85% 100% 95%
DeJean (n=34) 97% 97% 92% 76% 83% 97% 97% 97%
Helms (n=42) 100% 100% 91% 82% 85% 100% 86% 98%
Korematsu (n=43) 93% 98% 93% 74% 86% 98% 98% 100%
MS Total 97% 98% 91% 76% 84% 96% 94% 98%
Total 99% 99% 94% 88% 90% 98% 98% 98%
Source: Family Survey, Feb-March 2018 (N=1,447). Represents the proportion who responded “Agree” or “Strongly Agree”. The response option “I don’t know” was excluded from analysis. Actual survey items said “expanded learning program” instead of “ELP”.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
54
Table 13. Staff Survey: Number of Responses by Site and Lead Agency
Lead Agency/ Program Site
Supervisor Principal
Lead Teacher
School Day
Teacher
School Counselor
Group Leader
Other
Bay Area Community Resources
Bayview (n=7) 1 1 1 2 0 0 2
Chavez (n=12) 1 1 0 10 0 0 0
DeJean (n=12) 1 1 1 8 0 0 1
Dover (n=31) 1 1 1 21 2 0 5
Helms (n=25) 1 1 0 17 2 0 4
Highland (n=16) 1 1 1 10 0 1 2
Korematsu (n=21) 1 0 0 19 0 0 1
Lincoln (n=14) 1 1 1 6 0 1 4
Mira Vista (n=13) 1 1 1 7 0 1 2
Nystrom (n=23) 1 1 0 15 0 5 1
Riverside (n=10) 1 0 1 3 0 5 0
Stege (n=9) 1 0 0 5 0 0 3
Washington (n=9) 1 1 1 5 0 0 1
Wilson (n=26) 1 0 1 21 0 1 2
Love. Learn. Success.
Fairmont (n=11) 1 1 1 6 0 0 2
Ford (n=11) 1 1 1 7 0 0 1
Grant (n=31) 1 1 1 21 0 6 1
King (n=15) 1 1 1 11 0 0 1
Lake (n=24) 1 1 1 16 0 0 5
Montalvin (n=24) 1 1 1 11 2 4 4
Verde (n=3) 1 1 0 1 0 0 0
YMCA of the East Bay
Coronado (n=18) 1 1 1 10 1 0 4
Crespi (n=23) 1 1 1 18 0 0 2
Downer (n=23) 1 1 1 20 0 0 0
Murphy (n=13) 1 1 1 10 0 0 0
Peres (n=14) 1 1 1 4 0 1 6
Sheldon (n=16) 1 1 1 9 0 3 1
Tara Hills (n=21) 1 1 1 16 0 0 2
Total 28 24 22 309 7 28 57
Source: Staff survey, Feb-March 2018 (N=475).!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
55
Table 14. Staff Survey: Number of Visits to the Expanded Learning Program
Lead Agency/ Program % Who visited at
least 5 times % Who visited less
than 5 times % Who did not visit % Don't Know
Bay Area Community Resources
Bayview (n=7) 43% 14% 43% 0%
Chavez (n=12) 45% 27% 18% 9%
DeJean (n=12) 42% 42% 17% 0%
Dover (n=31) 40% 27% 33% 0%
Helms (n=25) 42% 29% 29% 0%
Highland (n=16) 56% 25% 19% 0%
Korematsu (n=21) 38% 29% 33% 0%
Lincoln (n=14) 50% 36% 14% 0%
Mira Vista (n=13) 69% 23% 8% 0%
Nystrom (n=23) 70% 17% 9% 4%
Riverside (n=10) 89% 11% 0% 0%
Stege (n=9) 56% 22% 22% 0%
Washington (n=9) 44% 22% 22% 11%
Wilson (n=26) 35% 35% 31% 0%
Love. Learn. Success.
Fairmont (n=11) 55% 18% 27% 0%
Ford (n=11) 55% 36% 9% 0%
Grant (n=31) 42% 23% 35% 0%
King (n=15) 73% 13% 13% 0%
Lake (n=24) 46% 42% 13% 0%
Montalvin (n=24) 43% 35% 22% 0%
Verde (n=3) 100% 0% 0% 0%
YMCA of the East Bay
Coronado (n=18) 56% 25% 19% 0%
Crespi (n=23) 14% 36% 50% 0%
Downer (n=23) 27% 18% 36% 18%
Murphy (n=13) 50% 25% 25% 0%
Peres (n=14) 38% 23% 38% 0%
Sheldon (n=16) 69% 6% 25% 0%
Tara Hills (n=21) 52% 19% 29% 0%
Total 48% 26% 25% 2%
Source: Staff survey, Feb-March 2018 (N=475).
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
56
Table 15. Staff Survey: Relationship Between School and Expanded Learning Program
Lead Agency/ Program
There is a strong partnership between the
ELP and the school.
The Site Supervisor keeps administration informed of important decisions and issues related to
program policy.
Teachers are willing to collaborate with the ELP
staff.
ELP staff are responsive to ideas and suggestions
from school staff.
ELP staff reach out to teachers to identify the
needs of students.
School staff are encouraged to visit the
program.
Bay Area Community Resources
Bayview (n=7) 57% 71% 43% 57% 57% 71% Chavez (n=12) 75% 67% 92% 75% 58% 67% DeJean (n=12) 92% 83% 75% 92% 92% 83% Dover (n=31) 81% 84% 81% 84% 55% 74% Helms (n=25) 84% 72% 72% 76% 56% 64% Highland (n=16) 94% 94% 88% 94% 81% 81% Korematsu (n=21) 48% 33% 48% 52% 33% 48% Lincoln (n=14) 71% 86% 79% 79% 71% 79% Mira Vista (n=13) 92% 85% 85% 85% 77% 77% Nystrom (n=23) 91% 74% 87% 87% 87% 83% Riverside (n=10) 100% 90% 100% 100% 80% 90% Stege (n=9) 100% 100% 56% 89% 89% 100% Washington (n=9) 89% 89% 89% 100% 67% 89% Wilson (n=26) 65% 54% 62% 58% 54% 32% Love. Learn. Success.
Fairmont (n=11) 73% 91% 64% 91% 64% 82% Ford (n=11) 100% 91% 91% 73% 45% 73% Grant (n=31) 94% 77% 81% 77% 68% 81% King (n=15) 73% 73% 87% 73% 60% 40% Lake (n=24) 83% 92% 83% 75% 58% 88% Montalvin (n=24) 67% 63% 79% 63% 54% 46% Verde (n=3) - - - - - - YMCA of the East Bay
Coronado (n=18) 76% 67% 72% 83% 50% 56% Crespi (n=23) 39% 48% 65% 39% 45% 43% Downer (n=23) 43% 35% 41% 43% 22% 22% Murphy (n=13) 38% 23% 69% 62% 46% 23% Peres (n=14) 64% 71% 50% 64% 36% 71% Sheldon (n=16) 81% 75% 75% 81% 81% 63% Tara Hills (n=21) 90% 86% 95% 90% 76% 62% Total 75% 71% 74% 74% 60% 64%
Source: Staff survey, Feb-March 2018 (N=475). Represents the proportion who responded “Agree” or “Strongly Agree”. Other response options were “Disagree”, “Strongly Disagree” and “I don’t know”. Actual survey items said “expanded learning program” instead of “ELP”. Data not provided for sites with 5 or fewer responses to maintain confidentiality. !
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
57
Table 16. Staff Survey: Relationship Between School and Expanded Learning Program (Continued)
Lead Agency/ Program
ELP staff follow through with the commitments they make to school
staff.
ELP staff transmit important information
about children and families to appropriate school staff
in a timely fashion.
ELP staff take care of the space the school
provides for the program.
Curriculum and instruction in the ELP
reinforce concepts taught during the school
day.
The ELP has enough capacity to serve all interested students.
The program is well coordinated with other
after school activities at the school.
Bay Area Community Resources
Bayview (n=7) 57% 71% 86% 43% 57% 71% Chavez (n=12) 67% 67% 83% 67% 33% 75% DeJean (n=12) 92% 92% 100% 67% 58% 92% Dover (n=31) 77% 65% 74% 68% 55% 80% Helms (n=25) 68% 64% 80% 56% 64% 80% Highland (n=16) 81% 81% 94% 88% 56% 94% Korematsu (n=21) 55% 38% 48% 33% 33% 43% Lincoln (n=14) 79% 79% 93% 50% 64% 64% Mira Vista (n=13) 77% 85% 100% 69% 54% 77% Nystrom (n=23) 82% 70% 78% 74% 52% 83% Riverside (n=10) 90% 80% 100% 100% 100% 90% Stege (n=9) 100% 100% 100% 78% 89% 100% Washington (n=9) 89% 89% 100% 78% 89% 100% Wilson (n=26) 52% 52% 54% 46% 35% 65% Love. Learn. Success.
Fairmont (n=11) 100% 64% 73% 73% 82% 55% Ford (n=11) 82% 73% 91% 55% 80% 100% Grant (n=31) 81% 74% 90% 81% 90% 87% King (n=15) 60% 60% 47% 40% 60% 67% Lake (n=24) 63% 50% 58% 46% 58% 67% Montalvin (n=24) 61% 71% 67% 71% 63% 67% Verde (n=3) - - - - - - YMCA of the East Bay
Coronado (n=18) 67% 44% 44% 39% 44% 56% Crespi (n=23) 35% 35% 65% 30% 65% 43% Downer (n=23) 26% 26% 39% 22% 17% 26% Murphy (n=13) 31% 42% 54% 23% 31% 62% Peres (n=14) 57% 50% 57% 64% 14% 50% Sheldon (n=16) 81% 81% 81% 75% 31% 60% Tara Hills (n=21) 75% 76% 81% 48% 43% 86% Total 68% 63% 73% 57% 55% 70%
Source: Staff survey, Feb-March 2018 (N=475). Represents the proportion who responded “Agree” or “Strongly Agree”. Other response options were “Disagree”, “Strongly Disagree” and “I don’t know”. Actual survey items said “expanded learning program” instead of “ELP”. Data not provided for sites with 5 or fewer responses to maintain confidentiality.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
58
Table 17. Staff Survey: Relationship Between School and Expanded Learning Program by Frequency
of Visits to the Expanded Learning Program
Source: Staff survey, Feb-March 2018 (N=475). Represents the proportion who responded “Agree” or “Strongly Agree”. Other response options were “Disagree”, “Strongly Disagree” and “I don’t know”. !
Survey Item Visited at least 5 times
Visited less than 5 times
Did not visit Don't Know if Visited
There is a strong partnership between the expanded learning program and the school. 84% 77% 62% 14%
The Site Supervisor keeps administration informed of important decisions and issues related to program policy. 85% 69% 50% 14%
Teachers are willing to collaborate with the expanded learning program staff. 81% 79% 63% 29%
Expanded learning program staff are responsive to ideas and suggestions from school staff. 90% 66% 54% 43%
Expanded learning program staff reach out to teachers to identify the needs of students. 72% 57% 45% 29%
School staff are encouraged to visit the program. 75% 63% 46% 43%
Expanded learning program staff follow through with the commitments they make to school staff. 82% 68% 43% 29%
Expanded learning program staff transmit important information about children and families to appropriate school staff in a timely fashion.
76% 63% 40% 43%
Expanded learning program staff take care of the space the school provides for the program.
78% 75% 63% 29%
Curriculum and instruction in the expanded learning program reinforce concepts taught during the school day.
69% 53% 41% 29%
The expanded learning program has enough capacity to serve all interested students.
60% 57% 44% 29%
The program is well coordinated with other after school activities at the school.
78% 73% 54% 43%
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
59
Table 18. Staff Survey: Communication Frequency Between Administrators and Site Supervisors
Source: Staff survey, Feb-March 2018 (n=220). Only includes staff who are in a leadership position (Principal, Site Supervisor, Expanded Learning Lead Teacher) and have visited the program 5 or more times.
!
Topic Never 1-2 times a semester Once a month
At least 2-3 times a month
Planning program content 21% 18% 41% 20%
Enrollment/Registration levels and policies 9% 30% 36% 24%
Staffing of program 18% 32% 23% 27%
Homework assignments 11% 35% 21% 33%
Curriculum concepts being taught in school 9% 29% 29% 33%
Issues related to classroom/shared space 9% 20% 32% 38%
The needs or progress of individual students 5% 23% 26% 47%
Student discipline issues/policies 5% 20% 21% 55%
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
60
APPENDIX D. ASSESSMENT AND GPA DATA
Table 19. Kindergarten Foundational Skills Assessment: Achievement Level; Participant vs. Non-
participant
Achievement Level Participant Non-participant
Needs intervention 0% 5%
Approaching standard 6% 11%
Meets or exceeds standard 94% 84%
Source: Kindergarten Foundational Skills Assessment Achievement Level. Participants (n=52), non-participants (n=660). Pearson chi-square: X2 (2, N = 712) = 4.320, p = .115.
Table 20. Kindergarten Foundational Skills Assessment: Achievement Level; Less than 100 days vs.
100 days or more
Achievement Level Less than 100 days 100 days or more
Needs intervention 0% 0%
Approaching standard 8% 4%
Meets or exceeds standard 92% 96%
Source: Kindergarten Foundational Skills Assessment Achievement Level. Participants that attended less than 100 days (n=24), participants that attended 100 days or more (n=28). Pearson chi-square: X2 (1, N=52) = .539, p = .463.
Table 21. STAR Early Literacy Assessment: Achievement Level; Participant vs. Non-participant
Achievement Level Participant Non-participant
Needs intervention 29% 38%
Approaching standard 26% 19%
Meets standard 11% 11%
Exceeds standard 34% 32%
Source: STAR Early Literacy Assessment Achievement Level, Spring 2018. Participants (n=76), non-participants (n=170). Pearson chi-square: X2 (3, N=246) = 2.416, p = .491.
Table 22. STAR Early Literacy Assessment: Achievement Level; Less than 100 days vs. 100 days or
more
Achievement Level Less than 100 days 100 days or more
Needs intervention 47% 25%
Approaching standard 27% 26%
Meets standard 7% 11%
Exceeds standard 20% 38%
Source: STAR Early Literacy Assessment Achievement Level, Spring 2018. Participants that attended less than 100 days (n=15), participants that attended 100 days or more (n=61). Pearson chi-square: X2 (3, N=76) = 3.396, p = .335.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
61
Table 23. STAR Early Literacy Assessment: Change in score; Participant vs. Non-participant
Participant Non-participant
Change in scale score from Fall 2017 to Spring 2018 169 180
Source: STAR Early Literacy Assessment change in scale score from fall 2017 to spring 2018. Participants (n=74), non-participants (n=161). Independent samples t-test: t(233) = -.726, p = .469
Table 24. STAR Reading Assessment: Achievement Level; Participant vs. Non-participant
Achievement Level Participant Non-participant
Urgent Intervention 54% 57%
Intervention Needed 14% 14%
On Watch 22% 19%
Meets Benchmark 9% 8%
Exceeds Benchmark 1% 2%
Source: STAR Reading Assessment Achievement Level, spring 2018. Participants (n=535), non-participants (n=1,031). Pearson chi-square: X2 (4, N=1,566) = 4.038, p = .401.
Table 25. STAR Reading Assessment: Achievement Level; Less than 100 days vs. 100 days or more*
Achievement Level Less than 100 days 100 days or more
Urgent Intervention 65% 49%
Intervention Needed 11% 15%
On Watch 17% 25%
Meets Benchmark 8% 9%
Exceeds Benchmark 0% 2%
Source: STAR Reading Assessment Achievement Level, spring 2018. Participants that attended less than 100 days (n=157), participants that attended 100 days or more (n=378). *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Pearson chi-square: X2 (4, N=535) = 13.159, p = .011.
Table 26. STAR Reading Assessment: Change in score; Participant vs. Non-participant
Participant Non-participant
Change in scale score from Fall 2017 to Spring 2018
69 68
Source: STAR Reading Assessment change in scale score from fall 2017 to spring 2018. Participants (n=477), non-participants (n=879). Independent samples t-test: t(1,354) = .110, p = .912.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
62
Table 27. SBAC English Language Arts Assessment: Achievement Level; Participant vs. Non-
participant*
Achievement Level Participant Non-participant
Standard Not Met 56% 49%
Standard Nearly Met 22% 22%
Standard Met 15% 20%
Standard Exceeded 7% 9%
Source: SBAC English Language Arts Assessment Achievement Level. Participants (n=2,238), non-participants (n=6,443). *The difference between participants and non-participants is statistically significant: Pearson chi-square: X2 (3, N=8,681) = 40.174, p < .01.
Table 28. SBAC English Language Arts Assessment: Achievement Level; Less than 100 days vs. 100
days or more*
Achievement Level Less than 100 days 100 days or more
Standard Not Met 62% 53%
Standard Nearly Met 19% 23%
Standard Met 14% 16%
Standard Exceeded 5% 8%
Source: SBAC English Language Arts Assessment Achievement Level. Participants that attended less than 100 days (n=821), participants that attended 100 days or more (n=1,417). *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Pearson chi-square: X2 (3, N=2,238) = 18.936, p < .01.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
63
Table 29. SBAC English Language Arts Assessment: Achievement Level; Participant vs. Non-
participant by Race/Ethnicity, Grades 4-6
Race/Ethnicity Achievement Level Participant Non-participant
Alaska Native/American Indian (n=24)
Standard Not Met 67% 67%
Standard Nearly Met 0% 11%
Standard Met 33% 17%
Standard Exceeded 0% 6%
Asian/Pacific Islander (n=539)
Standard Not Met 32% 31%
Standard Nearly Met 28% 22%
Standard Met 21% 29%
Standard Exceeded 19% 17%
Black/African American (n=914)
Standard Not Met 63% 65%
Standard Nearly Met 19% 17%
Standard Met 12% 13%
Standard Exceeded 6% 6%
Latino/Hispanic (n=3,115)
Standard Not Met 60% 54%
Standard Nearly Met 22% 22%
Standard Met 13% 17%
Standard Exceeded 4% 6%
White (n=377)
Standard Not Met 46% 35%
Standard Nearly Met 25% 15%
Standard Met 19% 25%
Standard Exceeded 10% 24%
Source: SBAC English Language Arts Assessment Achievement Level. Participants (n=1,286), non-participants (n=3,863). Students in grades 4-6 only.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
64
Table 30. SBAC English Language Arts Assessment: Achievement Level; Less than 100 days vs. 100
days or more by Race/Ethnicity, Grades 4-6
Race/Ethnicity Achievement Level Less than 100 days 100 days or more
Alaska Native/American Indian (n=5)
Standard Not Met 100% 50%
Standard Nearly Met 0% 0%
Standard Met 0% 50%
Standard Exceeded 0% 0%
Asian/Pacific Islander (n=131)
Standard Not Met 48% 26%
Standard Nearly Met 19% 31%
Standard Met 16% 23%
Standard Exceeded 16% 20%
Black/African American (n=284)
Standard Not Met 71% 59%
Standard Nearly Met 18% 19%
Standard Met 9% 14%
Standard Exceeded 1% 8%
Latino/Hispanic (n=800)
Standard Not Met 67% 57%
Standard Nearly Met 20% 24%
Standard Met 10% 15%
Standard Exceeded 3% 5%
White (n=66)
Standard Not Met 38% 49%
Standard Nearly Met 13% 30%
Standard Met 44% 11%
Standard Exceeded 6% 11%
Source: SBAC English Language Arts Assessment Achievement Level. Participants that attended less than 100 days (n=409), participants that attended 100 days or more (n=877). Students in grades 4-6 only.
Table 31. SBAC Math Assessment: Achievement Level; Participant vs. Non-participant*
Achievement Level Participant Non-participant
Standard Not Met 60% 56%
Standard Nearly Met 24% 25%
Standard Met 12% 11%
Standard Exceeded 5% 7%
Source: SBAC Math Assessment Achievement Level. Participants (n=2,270), non-participants (n=6,521). *The difference between participants and non-participants is statistically significant: Pearson chi-square: X2 (3, N=8,791) = 19.798, p < .01.!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
65
Table 32. SBAC Math Assessment: Achievement Level; Less than 100 days vs. 100 days or more*
Achievement Level Less than 100 days 100 days or more
Standard Not Met 68% 55%
Standard Nearly Met 18% 27%
Standard Met 9% 13%
Standard Exceeded 5% 4%
Source: SBAC Math Assessment Achievement Level. Participants that attended less than 100 days (n=844), participants that attended 100 days or more (n=1,426). *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Pearson chi-square: X2 (3, N=2,270) = 44.466, p < .01. Table 33. SBAC Math Assessment: Achievement Level; Participant vs. Non-participant by
Race/Ethnicity, Grades 4-6
Race/Ethnicity Achievement Level Participant Non-participant
Alaska Native/American Indian (n=24)
Standard Not Met 75% 61%
Standard Nearly Met 0% 22%
Standard Met 25% 11%
Standard Exceeded 0% 6%
Asian/Pacific Islander (n=539)
Standard Not Met 36% 34%
Standard Nearly Met 25% 32%
Standard Met 23% 20%
Standard Exceeded 16% 13%
Black/African American (n=914)
Standard Not Met 66% 66%
Standard Nearly Met 23% 23%
Standard Met 7% 8%
Standard Exceeded 4% 3%
Latino/Hispanic (n=3,115)
Standard Not Met 62% 58%
Standard Nearly Met 27% 27%
Standard Met 9% 11%
Standard Exceeded 2% 4%
White (n=377)
Standard Not Met 44% 34%
Standard Nearly Met 29% 29%
Standard Met 17% 16%
Standard Exceeded 10% 21%
Source: SBAC Math Assessment Achievement Level. Participants (n=1,286), non-participants (n=3,863). Students in grades 4-6 only.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
66
Table 34. SBAC Math Assessment: Achievement Level; Less than 100 days vs. 100 days or more by
Race/Ethnicity, Grades 4-6
Race/Ethnicity Achievement Level Less than 100 days 100 days or more
Alaska Native/American Indian (n=5)
Standard Not Met 100% 50%
Standard Nearly Met 0% 0%
Standard Met 0% 50%
Standard Exceeded 0% 0%
Asian/Pacific Islander (n=131)
Standard Not Met 50% 31%
Standard Nearly Met 19% 28%
Standard Met 19% 24%
Standard Exceeded 13% 17%
Black/African American (n=284)
Standard Not Met 73% 64%
Standard Nearly Met 20% 24%
Standard Met 5% 8%
Standard Exceeded 1% 5%
Latino/Hispanic (n=800)
Standard Not Met 71% 58%
Standard Nearly Met 22% 29%
Standard Met 6% 10%
Standard Exceeded 1% 2%
White (n=66)
Standard Not Met 31% 49%
Standard Nearly Met 31% 28%
Standard Met 19% 17%
Standard Exceeded 19% 6%
Source: SBAC Math Assessment Achievement Level. Participants that attended less than 100 days (n=409), participants that attended 100 days or more (n=877). Students in grades 4-6 only.
Table 35. GPA 2018: Participant vs. Non-participant
Participant Non-participant
GPA 2018 2.52 2.58
Source: WCCUSD GPA records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=567), non-participants (n=2,322). Independent samples t-test: t(2,887) = -1.347, p = .178.
Table 36. GPA 2018: Less than 100 days vs. 100 days or more*
Less than 100 days 100 days or more
GPA 2018 2.44 2.68
Source: WCCUSD GPA records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants that attended less than 100 days (n=376), participants that attended 100 days or more (n=191). *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Independent samples t-test: t(565) = -2.817, p = .005.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
67
APPENDIX E. ENGLISH LEARNER AND RECLASSIFICATION DATA
Table 37. English Learner Status: Participant vs. Non-participant
Participant Non-participant
English Learner 41% 43%
Non-English Learner 59% 57%
Source: WCCUSD demographic records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=3,477), non-participants (n=11,436). Non-English Learner includes students classified as “English or American Sign Language Only”, “Initial Fluent English Proficient”, or “Reclassified Fluent English Proficient”.
Table 38. English Learner Reclassification Rate: Participant vs. Non-participant
Participant Non-participant
Reclassified in 2017-18 12% 10%
Still an English Learner 88% 90%
Source: WCCUSD demographic records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=1,625), non-participants (n=5,441). Percentages only include students who started the school year classified as an English Learner. Pearson chi-square: X2 (1, N=7,066) = 2.434, p =.119.
Table 39. English Learner Reclassification Rate: Less than 100 days vs. 100 days or more*
Less than 100 days 100 days or more
Reclassified in 2017-18 9% 13%
Still an English Learner 91% 87%
Source: WCCUSD demographic records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants that attended less than 100 days (n=596), participants that attended 100 days or more (n=1,029). Percentages only include students who started the school year classified as an English Learner. *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Pearson chi-square: X2 (1, N=1,625) = 6.050, p =.014.
!
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
68
APPENDIX F. SCHOOL ATTENDANCE DATA Table 40. School Attendance Rate: Participant vs. Non-participant*
Participant Non-participant
Percent of total school days student attended school 95% 93%
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=3,482), non-participants (n =11,499). *The difference between participants and non-participants is statistically significant: Independent samples t-test: t(14,979) = 12.373, p < .01.
Table 41. School Attendance Rate: Less than 100 days vs. 100 days or more*
Less than 100 days 100 days or more
Percent of total school days student attended school 93% 96%
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants that attended less than 100 days (n=1,310), participants that attended 100 days or more (n=2,172). *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Independent samples t-test: t(3,480) = -12.812, p < .01.
Table 42. School Chronic Absence Rate: Participant vs. Non-participant*
Participant Non-participant
Percent of students who were chronically absent
18% 26%
Percent of students who were not chronically absent 82% 74%
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=3,482), non-participants (n = 11,499). Chronic absence defined as an absent rate of 10% or higher of the total days enrolled. *The difference between participants and non-participants is statistically significant: Pearson chi-square: X2 (1, N=14,981) = 96.198, p < .01. Table 43. School Chronic Absence Rate: Participant vs. Non-participant within Race/Ethnicity*
Participant Non-participant
Alaska Native/American Indian 26% 28%
Asian/Pacific Islander* 8% 16%
Black/African American* 25% 39%
Latino/Hispanic* 18% 26%
White 15% 19%
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants (n=3,482), non-participants (n = 11,499). Chronic absence defined as an absent rate of 10% or higher of the total days enrolled. *Asian/Pacific Islander: The difference between participants and non-participants is statistically significant: Independent samples t-test: t(1,741) = -3.697, p < .01. *Black/African American: The difference between participants and non-participants is statistically significant: Independent samples t-test: t(2,721) = -7.118, p < .01. *Latino/Hispanic: The difference between participants and non-participants is statistically significant: Independent samples t-test: t(9,048) = -7.772, p < .01.
WCCUSD Expanded Learning Programs 2017-18 Evaluation Findings | December 2018
69
Table 44. School Chronic Absence Rate: Less than 100 days vs. 100 days or more*
Less than 100 days 100 days or more
Percent of students who were chronically absent
31% 10%
Percent of students who were not chronically absent 69% 90%
Source: WCCUSD attendance records; matched to participants (August 21, 2017 – June 7, 2018) per program attendance records; Participants that attended less than 100 days (n=898), participants that attended 100 days or more (n=1,948). *The difference between participants who attended 100 days or more and participants who attended less than 100 days is statistically significant: Pearson chi-square: X2 (1, N=3,482) = 244.550, p < .01.