TGYS Orientation DayGet ready for evaluation!
ROADMAP
• TGYS Goals• Reasons for Evaluation• Evaluation Plans
• Design• Measurement• Data Collection• Timelines
• Annual Results
TGYS GOALS
REDUCE YOUTH CRIME AND VIOLENCE
PREVENT CHILD ABUSE AND NEGLECT
• Improve school performance
• Increase adult bonding• Increase self-esteem• Increase life skills• Decrease bullying• Decrease ATOD use• Decrease delinquency• Decrease recidivism• Improve quality of early
care and educational programs
• Achieve developmental milestones
• Increase positive parenting skills
TGYS Programs
and Activities
REASONS FOR EVALUATION
I. Your Participants
II. Reportin
g
III. Funding
REASON I
What’s working well?
What isn’t?
Program refinement
REASON I
GPA
School Engagemen
t
REASON II Package and streamline reporting
Mean NStd.
Deviation
Desired Direction of
Change?Sig.
Effect Size
Perceived relevance of school work - PRE 3.42 26 0.98
Yes 0.26 0.21
Perceived relevance of school work - POST 3.63 26 0.86
Enjoyment of school - PRE 3.26 25 0.47Yes 0.20 0.29
Enjoyment of school - POST 3.40 25 0.53
Effort in school - PRE 3.53 25 0.61Yes 0.17 0.24
Effort in school - POST 3.68 25 0.63
Educational aspirations - PRE 2.92 25 1.34Yes 0.04* 0.42
Educational aspirations - POST 3.48 25 1.22
School Bonding Total - PRE 3.36 25 0.53Yes 0.04* 0.39
School Bonding Total - POST 3.57 25 0.58
REASON III Competitive edge for funding
INDIVIDUAL EVALUATION PLANS
Step 1: Choose an evaluation
design
Step 2: Decide on the
best measurement
tool
Step 3: Design a data collection and management protocol
EVALUATION DESIGN
Pre-Test
Your Progra
mPost-Test
EVALUATION DESIGN
Pre-Test Your Program Post-Test
Pre-Test Post-Test
Group One: Program Participants
Group Two: Waiting-list Participants
MEASUREMENT
AnticipatedOutcomes
Validated, Reliable, & Practical
Tools
MEASUREMENT
Outcomes: Corresponding ScaleLife Skills: ResilienceAdult Bonding: Perceived Social SupportSchool Performance: Direct School RecordsSchool Performance: School EngagementSelf-Efficacy: Self EfficacyBullying: Bullying, Fighting, & VictimizationAlcohol, Tobacco, and Other Drug Use: Use & AttitudesDelinquency: Attitudes toward DelinquencyRecidivism: Colorado Criminal Contacts/Re-offensesQuality of Early Care and Education: Teaching Strategies GOLDDevelopmental Milestones: Social CompetenceParenting Skills: Parenting Practices Interview
DATA COLLECTIONIt’s “FUN”damental!
Electronic Surveys Paper Surveys Data Management
DATA COLLECTIONELECTRONIC SURVEYS
Obtain a Link
Complete Submit
Example survey link:Alternatives 4 Youth_COSchBond.
DATA COLLECTIONELECTRONIC SURVEYS
DATA COLLECTIONPAPER SURVEYS
Step 1: Ordering
Step 2: Preparation
Step 3: Submission
Double-check your order
Note the return address
Note the ID format
DATA COLLECTIONPAPER SURVEYS
Step 1: Ordering
Step 2: Preparation
Step 3: Submission
• Survey condition
• Administrative Section
• Instructions
• Confidentiality
DATA COLLECTIONPAPER SURVEYS
Useful Data
Condition • Using
originals
• Stapling or hole punching
• Using pencils
DATA COLLECTIONPAPER SURVEYS
• Fill in pre, post
or follow-up test
• Fill out Org ID and Local ID from top to bottom
• Date
DATA COLLECTION
ALL SURVEYS
Useful Data
Condition
Admin.Section
DATA COLLECTION
ALL SURVEYS
Useful Data
Condition
Admin.Section
Instructions
• Read thoroughly
• Pick best answer
• Answering questions
• Birthdate
DATA COLLECTION
ALL SURVEYS
Useful Data
Condition
Admin.Section
Instructions
Confidentiality
• Testing space
• Survey handling
Step 1: Ordering
Step 2: Preparation
Step 3: Submission
Julie MaertensColorado State
UniversityCampus Delivery 1876Fort Collins, CO 80523
DATA COLLECTIONPAPER SURVEYS
Assign unique IDs
Keep track of each ID
Keep your log confidential
DATA MANAGEMENT
Pre-test Post-test
1001 1001
1003 1005
2002 2002
2003 2003
2014 2112 Sample Size = 3
DATA MANAGEMENT
DATA MANAGEMENT
Deadline Task Person in ChargeAugust 16 Order pre-test surveys
(expect 2-week turn-a-round)
Mark V.
September 3 Organize surveys: count, fill out admin section
Susan C.
September 22 Administer surveys to youth
Laura F.
September 29 Check surveys: bubbles filled-in, admin complete
Susan C.
December 1 Mail pre-test surveys to CSU (2 weeks prior to Dec 15 deadline)
Mark V.
DATA MANAGEMENT
DATA MANAGEMENT
Pretest DeadlineDecember 15, 2011
Posttest DeadlineJune 30, 2012
INDIVIDUAL EVALUATION PLANS
Evaluation Design
Measurement Tool
Data Collection Protocol & Timeline
RESULTS
RESULTS
RESULTS
Mean NStd.
Deviation
Desired Direction of
Change?Sig.
Effect Size
Perceived relevance of school work - PRE 3.42 26 0.98
Yes 0.26 0.21
Perceived relevance of school work - POST 3.63 26 0.86
Enjoyment of school - PRE 3.26 25 0.47Yes 0.20 0.29
Enjoyment of school - POST 3.40 25 0.53
Effort in school - PRE 3.53 25 0.61Yes 0.17 0.24
Effort in school - POST 3.68 25 0.63
Educational aspirations - PRE 2.92 25 1.34Yes 0.04* 0.42
Educational aspirations - POST 3.48 25 1.22
School Bonding Total - PRE 3.36 25 0.53Yes 0.04* 0.39
School Bonding Total - POST 3.57 25 0.58
THANK YOU!
Also:[email protected]
CSU Evaluation Office(970)491-3830 main(877)724-4111 fax
Questions?
