Download - Evaluation 101: After School Programs
![Page 1: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/1.jpg)
Evaluation 101: After School Programs
February 1, 2007
Region 3 After School Technical Assistance Center
Conference
![Page 2: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/2.jpg)
Evaluation 101
ASES Evaluation—student data and how to get it
Prop 49 evaluation requirements in SB638
Suggested Timeline
Roles and Responsibilities
![Page 3: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/3.jpg)
ASES Evaluation Annually by October 1 funded programs must
submit to CDE data on participating students Two years data required:
2006/2007 follow up/program year 2005/2006 baseline/prior year
Data submitted to CDE in Excel file Evaluation narrative
![Page 4: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/4.jpg)
Prop 49 Evaluation SB638 Programs must submit individual student data:
Program attendance – participation Regular school day attendance STAR test results
Program effectiveness measures (1 or more from list) STAR test data Positive behavioral change* Homework completion rates* Skill development* Additional measures to be determined by CDE
* Measured by ASP teachers or regular day teachers
![Page 5: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/5.jpg)
Prop 49 Evaluation SB638
State Board of Education will define performance measures by November 2007
Funded programs will comply with state evaluation requirements determined in collaboration with Advisory Committee
![Page 6: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/6.jpg)
Terms ASP: After School Program
CHKS: California Healthy Kids Survey
Participation: after school program attendance
Regular attendance: regular school day attendance
Baseline: prior year 2005/2006
Follow up: program year 2006/2007
Continuing students: those who participated the previous year in After School Program
![Page 7: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/7.jpg)
ASP Evaluation Keys to Success
Four distinctive types of student information needed
Build evaluation database throughout year Clear roles and responsibilities Good ongoing communication Document process and sources
![Page 8: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/8.jpg)
Evaluation data: Four Types Needed on Students
Participation in after school program: number of days attended program in the school year
Regular school attendance: days enrolled and days absent in the school year
California Standards Test proficiency level math and language arts
California Healthy Kids Survey questions at entry and exit from program (4 questions)
![Page 9: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/9.jpg)
Sources of DataData Source
Participation of students in After School Program
ASP Program staff gather and record throughout year
Regular school attendance baseline and follow up
District data system for current and prior years; prior year ASES evaluation for continuing participants
California Standards Test results on math and language arts proficiency
District test director or curriculum director receives CST data annually
California Healthy Kids Survey 4 questions
ASP Program staff gather from students at entry and exit
![Page 10: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/10.jpg)
Sample Excel File in Folder
![Page 11: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/11.jpg)
Who Needs to be involved? ASP director and site staff
District data managers, or evaluation department, or fiscal department with regular school attendance information on students
District testing and curriculum staff who obtain the CST results annually and disseminate them to schools
Staff with expertise in Excel (or database software)
![Page 12: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/12.jpg)
Tips
Create a roster of key people involved in ASP evaluation with their phone numbers, faxes and email addresses.
Do ASP evaluation update every 2-3 months to keep all informed on progress. Will raise ASP evaluation visibility
Immediately thank via email and/or phone call anyone who gives you data
Write it down – for next year!
![Page 13: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/13.jpg)
Participation Data
![Page 14: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/14.jpg)
Participation Data
Prior to program beginning: Decide on how to gather and record program attendance:
after school program software, Excel spread sheets or Access database
Find last year’s evaluation report in Excel format to get baseline data (ongoing programs)
Decide on roles and responsibilities for data entry Who records attendance Who enters data into computer
![Page 15: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/15.jpg)
Participation Data
Record child’s name (first and last), grade, school, Student ID Number and days attended by month
As new children enter, record them in the data system on an ongoing basis
Continuing participants, gather baseline for participation, regular school attendance and CST from prior year’s Excel file
![Page 16: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/16.jpg)
Tips
Keep track of Student ID Numbers – they are the key to obtaining regular school attendance and CST results
Get child’s name as they are registered in school – no nicknames. James Jones not JJ Jones or Jimmie Jones
Don’t record attendance in Word , you will need to be able to add figures and sum them up
![Page 17: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/17.jpg)
Regular School Attendance
![Page 18: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/18.jpg)
Regular School Attendance
Data needed: Days enrolled, days absent current school year
2006/07 and prior school year 2005/06
Where is it kept: District data system Link by Student ID Number and school
![Page 19: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/19.jpg)
Regular School Attendance
What you need to do: Identify where data is kept in district Contact person who can get you access to data
and what you will need: initial information query in fall
Contact in March to obtain baseline prior year attendance for students new to program
Contact in June to obtain current year attendance for all participants
![Page 20: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/20.jpg)
Tips
Don’t be surprised if there is no prior year attendance for new participants – student mobility means students may be new to the school in program year
Check data for new participants in March for reasonableness Days enrolled should be around 170 (150-181) Days absent should be between 1 and 20; rarely
over 20
![Page 21: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/21.jpg)
California Standards Tests
![Page 22: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/22.jpg)
California Standards Tests Results available July or August following end of school
year
Math score and Language Arts Score
Scores are 1 through 5 or Far Below Basic, Below Basic, Basic, Proficient and Advanced
Best source for student demographics: gender, ethnicity, English Language Learner Status
![Page 23: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/23.jpg)
California Standards Tests Data needed
Proficiency level in math and language arts for program year (spring 2007) and baseline year (spring 2006)
Demographics on students Where it is kept
Comes to district testing staff July or August on CD ROM from state testing contractor
For continuing ASP students, baseline data is on prior year evaluation report to CDE
For new ASP students, baseline data is prior year CST results 2005/2006
![Page 24: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/24.jpg)
California Standards Tests What you need to do:
Identify test contact person in district Contact person who can get you access to test data:
initial information query fall Contact in March to obtain baseline prior year
proficiency for students new to program Submit file with id numbers of new students by school with
data request Contact in June to remind them of upcoming
summer request Submit file with id numbers of all participating students in
summer for new test data and new demographics
![Page 25: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/25.jpg)
Tips
• Don’t be surprised if there is no prior year CST results for new participants – student mobility means students may be new to the school in program year
• Keep in touch with testing person for district, sometimes data comes in early
• Don’t wait until the summer, many people are not working in summer months!
![Page 26: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/26.jpg)
California Healthy Kids Survey
![Page 27: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/27.jpg)
California Healthy Kids Survey
Train staff to administer survey when program begins and when students leave the program or near the close of school year
Enter initial (pre) surveys in evaluation database as they come in.
![Page 28: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/28.jpg)
Survey Questions1. In the past 30 days, how often have you wanted to go to school? (Never,
one or two times, About once a week, A few times a week, Everyday) 2. In the past 30 days, how often have you studied for a test? (Never, one or
two times, About once a week, A few times a week, Everyday) 3. In the past 30 days, how often have you felt unsafe at school? (Never, one
or two times, About once a week, A few times a week, Everyday) 4. In the past 30 days, how often have your parents asked you about school?
(Never, one or two times, About once a week, A few times a week, Everyday)
![Page 29: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/29.jpg)
Tips
Be sure to gather pre surveys for students who enter the program during the year.
Train staff to look for red flags if a student reports they don’t feel safe, for example.
Gather the post follow up surveys (same 4 questions as on entry to program) beginning about one month before the program ends.
Enter into database.
![Page 30: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/30.jpg)
Master timeline
Month before program starts
Fall
Decide on database format Set up participation data entry
system Train staff on California Healthy Kids
Survey and attendance procedures Find last year’s evaluation report
Excel files Identify person in district with regular
school attendance data and CST test scores and hold initial meeting
![Page 31: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/31.jpg)
Master timeline, continued
October through March
Check to see that you can generate list of students by school with id numbers and total days attended for first three months or six months
Check that number of initial California Healthy Kids Surveys in database match number of participants
Build baseline for ongoing students using ASP participation, regular school attendance and CST results from previous evaluation Excel file
![Page 32: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/32.jpg)
Master timeline, continued
May of school year
Meet with district attendance data contact; provide list of id numbers of participating students; obtain prior year attendance for new students
Review completeness of database, look for holes on continuing students and fill them
![Page 33: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/33.jpg)
Master timeline, continued
June
July
August
Request current year attendance data for all participants, giving school list of students with ID numbers
Request CST data and demographics from district test contact
Finalize Excel file with Student ID, days of ASP attendance; regular school attendance; CHKS answers
![Page 34: Evaluation 101: After School Programs](https://reader036.vdocuments.us/reader036/viewer/2022062806/56814f59550346895dbd0785/html5/thumbnails/34.jpg)
Master timeline, final
September Add test scores from CST to Excel files
Complete narrative Submit to CDE