super*vision training program
DESCRIPTION
Super*Vision Training Program. Evaluation Methodology. Jing Wan. Components. Introduction Logic model Evaluation methods(Design, data collection, analysis) Communication & dissemination. Introduction. Evaluation questions Formative evaluation Summative evaluation - PowerPoint PPT PresentationTRANSCRIPT
Super*Vision Training Program
Evaluation Methodology
Jing Wan
Components
Introduction Logic model Evaluation methods(Design, data
collection, analysis) Communication & dissemination
Introduction
Evaluation questionsFormative evaluationSummative evaluation Data collection methods & MeasuresFormative evaluationSummative evaluation(Qualitative measures+ quantitative measure) Communication & Dissemination
Program Analysis Overview of logic model
Program Analysis
Program Analysis
Inputs:Training structure & set-upTraining processTraining contentTrainerTraineeSupportive environment
Program Analysis
Outputs & OutcomesProgram products & activitiesChanges in trainees’ KSAImproved workforce & organizational performance
Evaluation Methods
Evaluation questions Evaluation design Data collection methods
Evaluation questions
Related to inputsIs training content relevant to the trainees’ job assignment?How satisfied are the trainees with their involvement in the training session? Related to outputsDoes the training program provide the planned assignments?
EQ
Related to outcomes/impactsDid the trainees experience any changes in their KSA, and job performance?What are the effects of the training program on each department of DHSMV (eg. Improved work efficiency, enhanced networking, and partnerships)?
Evaluation Design
Formative (process) evaluation designHow well the training program matches the evaluation theory behind its design, what the program actually does during the process. Summative (outcome) evaluation
designHow the training program is related to changes in the trainees, whether the program causes the desired changes.
Data collection methods
Formative (process) evaluationQuantitative measure: What to measure: trainees training experienceHow to measure: after training surveyFrom whom collected: traineesHow to analyze: statistical analysis for quantitative data
Example 1
Example 2
Data collection Cont.
Formative (process) evaluationQualitative measure:What to measure: the training meets the trainees’ expectationHow to measure: after training surveyFrom whom collected: traineesHow to analyze: content analysis for qualitative measure
Example 3
Data collection cont.
Summative (outcome) evaluationQualitative & quantitative measuresWhat to measure: short-term outcomesHow to measure: follow-up questionnaire & telephone interviewFrom whom collected: trainees & subordinatesHow to analyze: statistical analysis & content analysis
Example 4
Example 5
Communication & Dissemination
Communication challengesGeneral evaluation anxiety (results can affect decisions)DisengagementManagement operating style (resistant to change, rapid staff turnover, dysfunctional information-sharing system)
CD cont.
Communication methodsWritten reporting• Final evaluation report• Progress reportsVerbal presentations• Debriefing meetings Bring together stakeholders to present key evaluation findings and recommendations or other evaluation components.
ECP
ECP cont.
EMP
EMP cont.
