evaluation considerations from 38,000 feet if you don’t know where you are going, any road will...
TRANSCRIPT
Evaluation Considerations from 38,000 feet
if you don’t know where you are going, any road will get you there”
Yogi Berra
Peggy Smith, MA, Ph.D. Teen Health Clinics
Baylor College of MedicineBen Taub Hospital1504 Taub Loop
Houston, Texas 77030713-873-3601
Whether simple or complex the role of evaluation is to:
• Demonstrate that outcomes did not happen by chance
• Suggest that results can be repeated using chosen program– reliability
• Suggest that you are measuring what you said you would measure--validity
Types of evaluations
• Longitudinal• Cross-sectional• Quantative• Qualitative• Descriptive• Parametric
What ever type is chosen
• The selected outcome should relate to intervention
• The problem that is being addressed should drive the evaluation
• A simple but thoughtful evaluation is preferred to a more complicated approach.
• What you do is related to your objective
Clarification of funder and staff expectations
• Objectives• Variables• Interventions• Dosage• Follow-up – longitudinal versus cross
sectional• What to measure and how to measure it
Entry into Evaluation Process
• Intake Form- Completed at program entry.• Limited changes can be made with prior
approval. Forms need to be consistent.• Fill out completely. • Make forms obviously different so they are
kept in order. •
Tracking Clients
• Assign to each participant an unique ID number.
• ID numbers can be any combination of numbers or letters specific to a participant.
• If you re-enroll a participant for a second year, place an X in front of their ID number. XX for year three.
Quality Assurance Issues
• Tracking forms must be consistent. Participants can’t check smoking on one form and not on the next.
• Case managers must compare forms and make corrections before mailing to evaluator.
• Suggest participants fill out qualitative section of the form. Why it is important?
• How do you track participant progress?
Written Policies and Protocols
• What system is in place to check for completeness? Who is ultimately responsible?
• What is expected from participants in order to be in your program?
• How do you tell when someone has completed your program?• How/where you store consent forms. • What your policy is for obtaining consent forms from all
minors.
Peggy Smith, MA, Ph.D. Teen Health Clinics
Baylor College of MedicineBen Taub Hospital1504 Taub Loop
Houston, Texas 77030713-873-3601