![Page 1: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/1.jpg)
1
Patti P. Phillips, Ph.D.205-678-8101www.roiinstitute.net
BOTTOMLINEThe
ROIon
![Page 2: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/2.jpg)
Objectives
ØFivelevelsofevaluationthatserveasthefoundationoftheROIMethodology
ØSixtypesofresultsthatrepresentthechainofimpact
ØTenstepsoftheROIMethodologyprocessmodel
ØTwelveguidingprinciplesthatsupporttheROIMethodology
2
![Page 3: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/3.jpg)
Stayuntiltheend
![Page 4: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/4.jpg)
4
Whyfocusonresults?
![Page 5: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/5.jpg)
Whyfocusonresults?
• Investmentsbecomebusinessdecisions.
• Fundingisprotectedandenhanced.
• Programscancontinueandexand.• Businesspartnershipsaredeveloped.
• Supportincreases.• Seatat“the”tablebecomesavailable.
5
![Page 6: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/6.jpg)
6
Lackofresultsleadsto…
![Page 7: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/7.jpg)
Failuretodemonstrateresultsleadsto:
1. Investments being made on perception not evidence.2. Middle managers ceasing to support investments in learning.3. Diminishing commitment for learning and talent development.4. Waning influence.5. Curtailing of budgets and funding.6. Diminishing team satisfaction.
7
![Page 8: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/8.jpg)
Howdoyoudefineresults?
8
![Page 9: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/9.jpg)
Levels of Evaluation Measurement Focus
0. Inputs and Indicators The input into the project in terms of scope, volume, efficiencies, costs Participants, Hours, Costs, Timing
1. Reaction & Perceived Value
Reaction to the project or program, including the perceived value
Relevance, Importance, Usefulness, Appropriateness, Intent to use, Motivation to take action
2. Learning & Confidence
Learning to use the content and materials, including the confidence to use what was learned
Skills, Knowledge, Capacity, Competencies, Confidence, Contacts
3. Application & Implementation
Use of content and materials in the work environment, including progress with actual items and implementation
Extent of use, Task completion, Frequency of use, Actions completed, Success with use, Barriers to use, Enablers to use
4. Impact and Consequences
The consequences of the use of the content and materials expressed as business impact measures
Productivity, Revenue, Quality, Time, Efficiency, Customer Satisfaction, Employee Engagement
Typical Measures
5. ROI Comparison of monetary benefits from program to program costs
Benefit-Cost Ratio (BCR), ROI%, Payback Period
![Page 10: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/10.jpg)
The ROI Calculation
BCR =Program BenefitsProgram Costs
ROI =Net Program Benefits
Program CostsX 100
![Page 11: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/11.jpg)
BCR =$750,000$425,000
ROI = X 100$750,000 - $425,000$425,000
Try it!
![Page 12: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/12.jpg)
BCR =$750,000$425,000
= 1.76:1
ROI = X 100$750,000 - $425,000$425,000
= 76%
Try it!
![Page 13: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/13.jpg)
ROI Institute and ATD research show that the data CEOs receive are not demonstrating they are getting what they want out of their talent investment. (N=96)
MeasuresCurrently Measure
Should Measure Importance
Inputs and Indicators 94% 86% 6
Efficiency 78% 82% 7
Reaction 53% 22% 8
Learning 32% 28% 5
Application 11% 61% 4
Impact 8% 96% 1
ROI 4% 74% 2
Awards 40% 44% 3
What CEOs Want
![Page 14: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/14.jpg)
2015 Measurement and Metrics Study by CLO Magazine
35.6% Use business data to demonstrate the impact of the training organization on the broader enterprise.
21.6% Use ROI for that purpose.
22.6 % Plan to implement ROI within twelve months.
9.7% Plan implementation in twelve to twenty-four months.
17.3% Plan implementation with no time frame.
(N=335)
71.2%
CLOs Use or Plan to Use ROI
![Page 15: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/15.jpg)
2016 i4cp/ROI Institute study indicates that impact and ROI studies are top of mind when it comes to human capital analytics projects. (N=317)
Impact and ROI are Top of Mind
12%
29%
32%
34%
57%
69%
Other(pleasespecify)...
ForecastingROI(e.g.forecastingtheROIofawellnessinitiative)
Convertingdatatomoney(e.g.jobengagementtomonetaryvalues)
Developingpredictivemodels(e.g.screeningtestpredictsretention)
Developingrelationshipsbetweenvariables(e.g.connectingabsenteeismto…
MeasuringtheimpactandROIofaprogram(e.g.measuringtheROIofleadership…
15
![Page 16: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/16.jpg)
Ultimately, the success of any program is based on whether it improves business results.Training Top 10 Hall of Fame – May 2017
![Page 17: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/17.jpg)
17
ROI Methodology
EvaluationPlanning
DevelopObjectives
ofSolution(s)
DataCollection Reporting
DevelopEvaluationPlansandBaselineData
CollectDataDuringSolutionImplementation
CollectDataAfterSolutionImplementation
CaptureCosts
ofSolution
CalculatingtheReturnonInvestment
IdentifyIntangibles
Develop,Report,andCommunicate
Results
• Reaction and Planned Action
• Learning and Confidence
• Application and Implementation
• Business Impact
• ROI• Input/Indicators
DataAnalysis
IsolatetheEffectsofSolution
ConvertDatato
MonetaryValues
![Page 18: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/18.jpg)
Business Alignment and Forecasting The ROI Methodology
Learning Needs
Preference Needs
Evaluation
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1Initial Analysis
Input Needs 0 Input Objectives Input0
Program
Alignment Model
![Page 19: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/19.jpg)
0%
10%
20%
30%
40%
50%
60%
1 2 3 4 5 6 7 8
32% 29%
5%
55%
37%
11% 13% 11%
Method1. Control Groups2. Trend Line Analysis3. Forecasting Methods4. Participant Estimates5. Manager Estimates6. Sr. Management Estimates7. Expert Input8. Customer Input* Survey of Users, N = 235
Isolation Techniques
![Page 20: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/20.jpg)
1. Report the complete story2. Conserve evaluation resources3. Use the most credible sources of data4. Choose the most conservative alternatives5. Isolate the effects of the program6. Assume no data, no improvement7. Adjust estimates for error8. Throw out the extreme and unsupported claims9. Use first year benefits for short-term programs10. Include fully-loaded costs11.Report intangible benefits12.Communicate results to all stakeholders
12GuidingPrinciplesensurereliabilityintheprocess.
![Page 21: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/21.jpg)
Reaction
Learning
Application
Impact
ROI
Isolate the Effects of the Program
Intangible Benefits
Inputs
The final story includes six types of data.
![Page 22: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/22.jpg)
VZLU1Leadership Core+Functional
ROI=95%
4.8outof5.0onoverallsatisfactionwithtraining
86%ofparticipantsIncreasedSkillsontheJob
21%
33%
44%
Lunch
Leadership Module
Small Group Exercises
Top3factorsaffectingimpact:
Oneononeswithdirects
ManagerSupport
Opportunitytopractice/apply
40%
20%
10%
Participants =152WORTHIT?
IMPACTEDTHEBUSINESS?
APPLIEDONTHEJOB?
LIKEIT?
CLIMATEFACTORS?
NetAddsaboveQuota*
12.0NetAddsaboveQuotaperpersondirectlyduetotraining
LEARNANYTHING?
98%ofgainednewandvaluableknowledgefortheirjob
*Graphshowstotalimprovementbeforeisolation
Forevery$1wespent,wemadeback$1.95
Used with permission from Verizon
![Page 23: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/23.jpg)
Sample Case StudiesTransnet Coach Business South AfricaSupervisor coaching program focused on improving beat rate and sales targets. ROI = 32.64%
PolyWrigton USAWork engagement intervention focused on controllable waste, rework, and engagement. ROI = 399%
Industrial Company GermanyGlobal sales training focused on improving sales. ROI = 161%
Baptist Hospital USASexual harassment prevention program focused on reduction in complaints, turnover, and bad behavior.ROI = 1051%
![Page 24: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/24.jpg)
What can you do?
![Page 25: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/25.jpg)
Moveyourpracticeforwardby…
25
ü Assessing your readiness for ROI
ü Identifying stakeholders and their data needs
ü Determining the purpose of your evaluation practice
ü Identifying programs suitable for ROI
ü Building capability in the ROI Methodology
![Page 26: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/26.jpg)
FreeTools
26
![Page 27: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/27.jpg)
27
2018 ROI Certification Events•Charlotte, NC – January 15-19
•Atlanta, GA – February 12-16
•Orlando, FL – February 26-March 2
•New Orleans, LA – March 5-9
•Seattle, WA – April 9-13
•Philadelphia, PA – April 23-27
•San Diego, CA – May 7-11
•Denver, CO – May 21-25
•Long Beach, MS – May 21-25
•Atlanta, GA – August 20-24
Visit www.roiinstitute.net/roi-certification for more details
![Page 28: The BOTTOMLINE on ROI - HRDQ-U · Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 ... business results. Training Top 10 Hall of Fame –May](https://reader030.vdocuments.us/reader030/viewer/2022040412/5f0830cd7e708231d420c9ff/html5/thumbnails/28.jpg)
28
Patti P. Phillips, Ph.D.205-678-8101www.roiinstitute.net
BOTTOMLINEThe
ROIon