measuring the effectiveness of training - myth or reality?
DESCRIPTION
The intangible nature of training coupled with the number oriented measure of ROI, makes 'Training ROI' a holy grail for learning and development professionals. Because of this, most organisations are reluctant to embark on an ambitious project to put in place a framework to measure the effectiveness of training. However, companies are now beginning to realise that ROI (in its usually accepted format) is not the only way of measuring the effectiveness of training. An effective framework for training starts with aligning all training activities to the business objectives of the organisation. This presentation shares some key insights based on the Kirkpatrick Model for evaluation of training effectiveness and shows the critical need for HR to play a strategic role in organisations today.TRANSCRIPT
![Page 1: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/1.jpg)
2 Aug 2013PMG Meeting, Singapore
Measuring the effectiveness of training – myth or reality?
![Page 2: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/2.jpg)
What is
EFFECTIVENESSof training?
![Page 3: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/3.jpg)
What are some key
CHALLENGESin organization training?
![Page 4: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/4.jpg)
Program design is not always linked to business objectives
Training may not be addressing the key needs of the organisation
Implication
![Page 5: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/5.jpg)
Lack of effective post-training follow up, reinforcement and measurement
Training effectiveness
cannot be measured
Implications
Learning design cannot be
adjusted in real time
Performance gap not closed
in time
![Page 6: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/6.jpg)
Participants are not clear about the business goals and strategic initiatives of
the company
Transfer of learning to the job may not happen
Implication
![Page 7: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/7.jpg)
You cannot manage what you cannot measure!
”“
![Page 8: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/8.jpg)
The
KIRPATRICKmodel for evaluation
![Page 9: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/9.jpg)
The Kirkpatrick Model for training evaluation
![Page 10: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/10.jpg)
Reaction
The Kirkpatrick Model for training evaluation
Smile sheetFeedback form
To what degree participants
react favourably to the learning
event
Level 1
![Page 11: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/11.jpg)
LearningReaction
The Kirkpatrick Model for training evaluation
Pre/Post testPerformance
test
Smile sheetFeedback form
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 2Level 1
![Page 12: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/12.jpg)
BehaviourLearningReaction
The Kirkpatrick Model for training evaluation
Superior f/backBehaviour obs.
Pre/Post testPerformance
test
Smile sheetFeedback form
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 3Level 2Level 1
![Page 13: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/13.jpg)
ResultsBehaviourLearningReaction
The Kirkpatrick Model for training evaluation
% increase in sales, profits
etc.
Superior f/backBehaviour obs.
Pre/Post testPerformance
test
Smile sheetFeedback form
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
![Page 14: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/14.jpg)
ResultsBehaviourLearningReaction
The Kirkpatrick Model for training evaluation
% increase in sales, profits
etc.
Superior f/backBehaviour obs.
Pre/Post testPerformance
test
Smile sheetFeedback form
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
Easy to implementUsually done
![Page 15: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/15.jpg)
ResultsBehaviourLearningReaction
The Kirkpatrick Model for training evaluation
% increase in sales, profits
etc.
Superior f/backBehaviour obs.
Pre/Post testPerformance
test
Smile sheetFeedback form
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
Difficult to implementNot usually done
Easy to implementUsually done
![Page 16: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/16.jpg)
How can this be
IMPLEMENTED
![Page 17: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/17.jpg)
How can this be
IMPLEMENTED(easily)?
![Page 18: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/18.jpg)
Start TNA with the end in mind – what are the organisation’s business
goals
1
![Page 19: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/19.jpg)
Translate business goals into indicators or metrics for departments
and individuals
2
![Page 20: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/20.jpg)
Identify the specific actions and behaviours that can positively impact
the business goals
3
Actions and behaviours
must be measurable
![Page 21: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/21.jpg)
Write learning objectives that can be drivers for these actions and
behaviours
4
![Page 22: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/22.jpg)
Build post-training activities during program design
5
Only 15% of what is learned during training will be applied on the job if it is not reinforced and monitored.
- Research by Robert Brinkerhoff
![Page 23: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/23.jpg)
ResultsBehaviourLearningReaction
Starting with the end in mind
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
![Page 24: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/24.jpg)
ResultsBehaviourLearningReaction
Starting with the end in mind
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
“Start with the end in mind”
“Identify indicators and behaviours”
“Design post training activities with program design and evaluation”
![Page 25: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/25.jpg)
ResultsBehaviourLearningReaction
Starting with the end in mind
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
1
“Start with the end in mind”
“Identify indicators and behaviours”
“Design post training activities with program design and evaluation”
![Page 26: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/26.jpg)
ResultsBehaviourLearningReaction
Starting with the end in mind
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
12 3
“Start with the end in mind”
“Identify indicators and behaviours”
“Design post training activities with program design and evaluation”
![Page 27: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/27.jpg)
ResultsBehaviourLearningReaction
Starting with the end in mind
To what degree targeted outcomes occur, as a
result of learning
event(s) and subsequent
reinforcement
To what degree participants apply what
they learned during training when they are back on the job
To what degree participants acquire the intended
knowledge, skills and attitudes based on their participation in
the learning event
To what degree participants
react favourably to the learning
event
Level 4Level 3Level 2Level 1
12 34 5
“Start with the end in mind”
“Identify indicators and behaviours”
“Design post training activities with program design and evaluation”
![Page 28: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/28.jpg)
What will this
ACHIEVE?
![Page 29: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/29.jpg)
Framework for measurement of
effectiveness of training
![Page 30: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/30.jpg)
Training aligned to business goals
![Page 31: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/31.jpg)
Ability to make real time changes to help
meet the business goals
![Page 32: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/32.jpg)
Measurable and Effective Training
![Page 33: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/33.jpg)
What is needed for the
SUCCESSof this model?
![Page 34: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/34.jpg)
Agreement on business measures is required even before the learning
intervention is designed
Other teams need to get involved in the entire learning process
Implication
![Page 35: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/35.jpg)
Participants (and their supervisors) need to be aware of the business goals, the objectives of the program and how it links to their job and
KPIs
Communication between
HR/Training and other teams is
critical
ImplicationsParticipants and supervisors must understand what will be measured
after training
Supervisors may need to be included in the learning and evaluation design
process
![Page 36: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/36.jpg)
Appropriate tools and processes for post training follow up and measurement need to be
put into place
Indicators and measures of behaviours and actions
need to be defined clearly and realistically
Implications
Process of measurement should be defined
![Page 37: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/37.jpg)
Monitoring mechanism / tools need to be available to check in real time if performance gaps are being closed and business goals are
being met
Business dashboard for training / learning & development is required for monitoring and tracking
Implication
![Page 38: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/38.jpg)
But the mostCRITICAL
NEEDis this…
![Page 39: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/39.jpg)
HR has to STOP playing a support role in the organisation!
![Page 40: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/40.jpg)
HR needs to start playing the role of a
strategic business partner in the organization
![Page 41: Measuring the Effectiveness of Training - Myth or Reality?](https://reader034.vdocuments.us/reader034/viewer/2022052521/54568fbeaf7959bc078b5800/html5/thumbnails/41.jpg)
Accord Business Solutions Pte. Ltd.146 Robinson Road, #07-01, Singapore 068909
Phone: +65 9298 9590Singapore | India | UAE
www.accordsingapore.com.sg | [email protected]
A presentation by Saurabh Anand