the roi of learning
TRANSCRIPT
1
2 Demonstrating the ROI of Learning
Welcome + Introduction
April 22, 2015
Brian Smith
Director of Enterprise
Marketing
3
A Robust Online Training Library
Content in BusinessLeadership, Management, Productivity, Software,
Communication, Career Development, and more.
3,400+ coursesAvg. 20 courses released per week
10,000+ hours of learningAvg. 5 hours released per day
138,000+ videosAvg. 750 released per week
Demonstrating the ROI of Learning
4
Britt Andreatta
Leadership consultant since 1989
PhD in Education, Leadership + Organizations
Professor and Dean at UC Santa Barbara
and Antioch University, teaching leadership
and success skills
lynda.com Member > Author > Director of Learning + Development
Please view my seven courses on lynda.com
Demonstrating the ROI of Learning
5
Agenda
Return on Investment (ROI)
Your Context
Results That Matter
The Amazing Value of Learning
The ROI Method and Process
Demonstrating the ROI of Learning
6
Sources
Demonstrating the ROI of Learning
7
Agenda
Return on Investment (ROI)
Your Context
Results That Matter
The Amazing Value of Learning
The ROI Method and Process
Demonstrating the ROI of Learning
8
History
Peter Drucker
“If you can’t measure it,
you can’t manage it.”
Demonstrating the ROI of Learning
9
Donald Kirkpatrick
Demonstrating the ROI of Learning
Four levels of
evaluation
10
Donald Kirkpatrick
Demonstrating the ROI of Learning
Results
Behavior
Transfer
Reaction
11
Jack Phillips
Demonstrating the ROI of Learning
The Father of ROI
12
ROI
Demonstrating the ROI of Learning
Did the learners
like the program?
Did the learners
learn the content?
Did the learners deploy
the learning on the job?
Did the learning impact
business results?
Did the learning
investment payoff?
Jack PhillipsROI
Impact
Implementation
Learning
Reaction/Satisfaction
13
The Math
Benefit-cost Ratio (BCR)
Return on Investment (ROI)
Time to payback
Works with any currency
$ ¥ ₹ €, etc.
Demonstrating the ROI of Learning
14
Benefit-cost Ratio (BCR)
BCR =Program Benefits
Program Costs
Economic indicator of accountability for results.
The benefit (return) of an investment is compared to
the cost; expressed as a ratio.
Demonstrating the ROI of Learning
(x:y)
15
BCR Example
$35,000
$20,000
It cost $20,000 to implement new learning program.
The learning resulted in $35,000 savings.
BCR = = 1.75
1.75:1Demonstrating the ROI of Learning
For every $1 spent, you received $1.75 in total benefit.
16
Return on Investment (ROI)
ROI =
(Program Benefit
Minus Program Costs)
Program Costs
Economic indicator of accountability for results.
The benefit (return) of an investment is divided by the
cost; expressed as percentage.
X 100
Demonstrating the ROI of Learning
%
17
ROI Example
ROI =¥35,000-¥20,000
¥20,000
It cost ¥20,000 to implement new learning program.
The learning resulted in ¥35,000 savings.
X 100 = .75
75%Demonstrating the ROI of Learning
For every ¥1 spent, you received ¥.75 in net benefit.
18
Time to Payoff
Payback = Program Costs
Program Benefits
Economic indicator of accountability for results.
Estimated time in which a program will break even (time
after that is added benefit); expressed in time.
X time unit
Demonstrating the ROI of Learning
Time
19
Payoff Example
Payback =₹20,000
₹35,000
It cost ₹20,000 to implement new learning program.
The learning resulted in ₹35,000 savings.
= .57
6.84 monthsDemonstrating the ROI of Learning
The program would pay for itself in 6.84 months.
x 12 = 6.84
20
Overwhelmed?
You only need to build
a compelling case for
your context.
Demonstrating the ROI of Learning
21
Agenda
Return on Investment (ROI)
Your Context
Results That Matter
The Amazing Value of Learning
The ROI Method and Process
Demonstrating the ROI of Learning
22
Reality Check
Is HR/L+D seen as a trusted and valued
business partner?
Do you have an established record of success?
Are you empowered to take intelligent risks
and make decisions?
Are you justifying an upcoming purchase or
evaluating one you have already implemented?
Demonstrating the ROI of Learning
23
Plenty of Options
Cost
Output
Time
Quality
Energy
Demonstrating the ROI of Learning
24
Hard Data
Demonstrating the ROI of Learning
Cost Output Time Quality Energy
Unit costsOverhead costsOperating costs
VariancesInsurance/legalPenalties/finesAccident costsSales expense
Units producedTons manufactured
Items assembledReports processes
Students graduatedGrants awardedTasks completedAccounts signed
Cycle timeResponse time
DowntimeOvertime
Processing timeSupervisory timeWork stoppages
ErrorsWaste
ReworkRejectsDefects
ShortagesFailures
Accidents
WaterFossil fuels
FoodMinerals
LandTrees
PollutionWaste
25
Plenty of Options
Cost
Output
Time
Quality
Energy
Demonstrating the ROI of Learning
Service
Creativity
Development
Culture
26
Soft Data
Demonstrating the ROI of Learning
Customer Service Creativity Development Culture
ImpressionServiceLoyalty
RetentionComplaints
Returns
New ideasInnovationRisk taking
SuggestionsCollaborationPartnership
Alliances
Job effectivenessCapability
PerformancePotential
PromotionsRequests for
transfer
TurnoverComplaintsGrievances
AbsenteeismTardiness
EngagementJob satisfaction
Loyalty
27
Case Study
Demonstrating the ROI of Learning
“lynda.com access
means there is a constant
learning resource for
employees. Sometimes it
only takes two minutes to
improve their knowledge,
rather than a full day in a
classroom to achieve the
same outcome.”
BENEFITS
• Complements live
training
• Lets employees quickly
find answers to questions
• Provides detailed reports
that inform future training
initiatives
ABOUT DENTSU AEGIS
Denstu Aegis Network helps
clients build relationships with
consumers through content
creation, digital creative
execution, market analysis,
and much more.
23,000 employees in 110
countries on 5 continents
28
Case Study
Demonstrating the ROI of Learning
“I honestly think this
hands-on learning is the
best way to learn
software. I’m glad UCLA
provides lynda.com to all
students. I think the more
software we know, the
more it prepares us for the
job market once we
graduate.”
BENEFITS
• Supports students in
any major
• Provides tools training
outside of class
• Encourages staff and
faculty professional
development
ABOUT UCLA
Founded in 1919, UCLA
offers 125 undergraduate
and graduate degrees in
business, engineering,
medicine and more.
35,000 students + 31,000
full- and part-time faculty and
administrative staff.
29
Case Study
Demonstrating the ROI of Learning
“If we’re struggling to
learn something, or if we’re
frustrated trying to get
something to work,
someone will see if a
lynda.com course exists.
With such a large amount of
content at an affordable
price, lynda.com was by far
our best bet.”
BENEFITS
• Provides convenient
on-demand solution
• Offers cost-effective and
current training
• Teaches wide range of
software, creative, and
business skills
• Helps employees learn at
their own level and pace
ABOUT EPA
Established in 1970, the EPA
enforces federal health and
environmental laws so that all
Americans are safe where
they live, learn and work.
17,000 employees located in
10 regional offices across US
30
Agenda
Return on Investment (ROI)
Your Context
Results That Matter
The Amazing Value of Learning
The ROI Method and Process
Demonstrating the ROI of Learning
31
Business Cents > Business Sense
What problem are you solving?
How can progress be measured?
Who are the stakeholders?
Who has what expectations?
Demonstrating the ROI of Learning
32
ROI vs. ROE
Demonstrating the ROI of Learning
Investment Expectation
JustificationEstimate cost and
benefit of investmentAlign your design, and
estimate cost and benefit
Evaluation
Isolate the effect of investment and
calculate its value
Demonstrate extent that expectations were met and
calculate the value
33
Measuring Success
Identify metrics that matter
Use data you can get
Increasing vs. Decreasing
Activity vs. Results
Learning vs. Performance
Demonstrating the ROI of Learning
34 Demonstrating the ROI of Learning
35 Demonstrating the ROI of Learning
36
Increasing X vs. Decreasing Y
Demonstrating the ROI of Learning
From A
To B by when
Time
From A
Time
To B by when
Increase productivity or decrease inefficiency?
Increase retention or decrease attrition?
Increase revenue or decrease waste?
37
Activity vs. Results
Lead and Lag Measures
Demonstrating the ROI of Learning
38 Demonstrating the ROI of Learning
Lead vs. Lag Measures
Lead Measure(activity)
Lag Measure(results)
measures goalpredictive of goal + influenceable
lose weight/sizediet + exercise
doing this… gets that
= LEVERAGE
39 Demonstrating the ROI of Learning
Examples
Lag Measure(results)
reduce equipment failure
reduce safety incidents
increase accuracy/efficiency
reduce churn
increase retention of talent
Lead Measure(activity)
routine maintenance
compliance with safety regs
time spent practicing
touchpoints with clients
coaching provided
40 Demonstrating the ROI of Learning
Implement Manager Training
Lag Measure(results)
Increase retention of talent
Decrease complaints to HR
Lead Measure(activity)
Regular one-on-ones
Provide coaching
Give autonomy
Support growth
Build cohesive teams
41 Demonstrating the ROI of Learning
ReactionObjectives
LearningObjectives
ImplementationObjectives
BusinessImpact
• Perceive program to be relevant to the job
• Perceive program to be relevant to job performance
• Perceive program to be value added
• Rate the program as effective
• Would recommend to others
• Understand the value of engagement and their influence
• Use skills and clarity coaching to develop employees
• Cultivate team cohesion and collaboration
• Improve personal effectiveness
Complete action plan
Show improvements in:• coaching skills• team management• personal
effectiveness
Identify barriers and enablers to application of knowledge/skills acquired
• Increased retention
• Increased productivity
• Reduced complaints to HR
• Reducedabsenteeism
ROI
Impact
Implementation
Learning
Reaction/Satisfaction
42
Learning + Performance Outcome
Demonstrating the ROI of Learning
ReactionObjectives
LearningObjectives
ImplementationObjectives
BusinessImpact
• Perceive program to be relevant to the job
• Perceive program to be relevant to job performance
• Perceive program to be value added
• Rate the program as effective
• Would recommend to others
• Understand the value of engagement and their influence on it
• Use skills and clarity coaching to develop employees
• Cultivate team cohesion and collaboration
• Improve personal effectiveness
Complete action plan
Show improvements in:• coaching skills• team management• personal
effectiveness
Identify barriers and enablers to application of knowledge/skills acquired
• Increased retention
• Increased productivity
• Reduced complaints to HR
• Reducedabsenteeism
43
Agenda
Return on Investment (ROI)
Your Context
Results That Matter
The Amazing Value of Learning
The ROI Method and Process
Demonstrating the ROI of Learning
44
Training + Learning
Demonstrating the ROI of Learning
training
learning
Not to scale!
45 Demonstrating the ROI of Learning
Training < ----------- ---------- > Learning
Specific event or activity Anywhere, any time
Designed from organization’s perspective Designed from learner’s perspective
Develop new skills, so can carry out specific behavior/process on own (prescriptive)
Absorb and retain information to be used in non-specific, unexpected future use (creative)
Given to; Organization tells/teaches/directs employee
Engaged with; Organization partners with learner to grow, develop, adapt
May have goal for compliance and/or risk reduction
Goal is to enhance potential and performanceof employee in ways that are meaningful
Employee can perceive content to be irrelevant or boring, so may be disengaged and unmotivated to pursue further learning
Employee perceives content as immediately useful and helpful, so active participant and motivated to pursue further learning
46
Benefits of Learning
Demonstrating the ROI of Learning
Active participation
Authentic motivation
Ongoing improvement
Employee engagement
Retention of talent
47
Motivation
Demonstrating the ROI of Learning
Autonomy opportunities to be self directed
Masteryopportunities to learn and grow
Purposecontribute to something meaningful
48
Continued
Improvement
Growth mindset
Dr. Carol Dweck
49 Demonstrating the ROI of Learning
Growth Mindsetleads to a desire to learn, so tends to:
Believe that skills can always improve with
hard work
See effort as a path to mastery and
therefore essential
Embrace challenges and see them as
opportunity to grow
See feedback as useful for learning and
improving
Views setbacks as a wake-up call to work
harder next time
Find lessons and inspiration in the success
of others
As a result, they reach ever-higher
levels of potential and performance.
Fixed Mindsetleads to a desire to look good, so tends to:
Believe that most skills are based on traits that
are fixed and cannot change
See effort as unnecessary; something to do
when you’re not good enough
Avoid challenges because could reveal lack
of skill; tends to give up easily
See feedback as personally threatening to
sense of self and gets defensive
View setbacks as discouraging; tends to blame
others
Feel threatened by the success of others;
may undermine others in effort to look good
As a result, they may plateau early and
achieve less than their full potential.
50
Engagement
How much people are
emotionally connected and
committed to their organizations,
and their willingness to go above
and beyond the expectations of
their jobs.
Demonstrating the ROI of Learning
51
Measuring Engagement
Demonstrating the ROI of Learning
1. I am proud to work for this company.
2. This company motivates me to go beyond what
I would in a similar role elsewhere.
3. I see myself still working here in 3 years.
4. I would recommend this company as a great place to work.
5. I rarely think about looking for a job at another organization.
52
53
In the US Around the World
Demonstrating the ROI of Learning
% of US Workers
Engaged 30%
Not engaged 52%
Actively disengaged18%
54 Demonstrating the ROI of Learning
55
The Power of Engagement
Demonstrating the ROI of Learning
A disengaged employee costs an organization
approximately $3,400 for every $10,000 of salary (Gallup).
Engaged employees are 127% more likely to be
A performers than C performers (McLean & Company).
Highly engaged organizations have the potential to
decrease employee turnover by 87% (Human Capital Institute).
56 Demonstrating the ROI of Learning
From “Making
the Case for
Employee
Engagement”
57
L+D is a key
driver of
employee
engagement
Demonstrating the ROI of Learning
58
The Role of L+D
Demonstrating the ROI of Learning
Have data that show how
L+D is directly related to:
increases in overall
employee engagement
increases in employee identification with organization’s values
increase in positive organization culture
59
Retaining Top Talent
Demonstrating the ROI of Learning
Cost to replace an employee is50% to 250% of annual salary+benefits!
SHRM’s “Cost of Turnover” Worksheet
• Lost productivity (position + others)
• Offboarding
• Recruiting > hiring > training
• Time to previous employee’s performance
60
ROI Calculator
Demonstrating the ROI of Learning
61
How to Maximize
the Benefits of
Learning
Demonstrating the ROI of Learning
62
Cultivate Potential
Demonstrating the ROI of Learning
Having the capacity to
become or develop into
something in the future.
Unrealized ability.
Performance = Potential + Support
63
Put Learners First
Demonstrating the ROI of Learning
What is their context?
What would make their work life immediately better or easier?
Design to solve their problems and maximize their potential.
64
Be a
Learning
Guide
Demonstrating the ROI of Learning
They will help you show ROI.
65
Agenda
Return on Investment (ROI)
Your Context
Results That Matter
The Amazing Value of Learning
The ROI Method and Process
Demonstrating the ROI of Learning
66
ROI Methodology
Demonstrating the ROI of Learning
Five Levels of Evaluation
V Model for Business Alignment
Process Model to Analyze Impact
Data Collection
67
Jack Phillips
ROI
Demonstrating the ROI of Learning
ROI
Impact
Implementation
Learning
Reaction/SatisfactionDid the learners
like the program?
Did the learners
learn the content?
Did the learners deploy
the learning on the job?
Did the learning impact
business results?
Did the learning
investment payoff?
68 Demonstrating the ROI of Learning
V Model
69
The ROI Process Methodology
Demonstrating the ROI of Learning
70 Demonstrating the ROI of Learning
71
Data Options
Demonstrating the ROI of Learning
Hard/soft data
Surveys/questionnaires
Tests
Observation
Interviews
Focus groups
Performance Records
72 Demonstrating the ROI of Learning
73
Isolating Effect of Program
Pre-test, post-test
Control group
Trend analysis
Use of expert studies/data
Estimation of impact by:• Participants
• Supervisors
• Leadership
Demonstrating the ROI of Learning
74 Demonstrating the ROI of Learning
Learn More About ROI
75
Learn More About Learning at lynda.com
Instructional Design Essentials series:
❯ The Neuroscience of Learning with Britt Andreatta
❯ Flipping the Classroom with Aaron Quigley
❯ Models of Instructional Design with Shea Hanson
❯ Needs Analysis with Jeff Toister
Running a Profitable Business with Jim Stice
Finance Fundamentals with Jim and Kay Stice
Demonstrating the ROI of Learning
76 Demonstrating the ROI of Learning
77 Demonstrating the ROI of Learning
78
Questions + Answers
Demonstrating the ROI of Learning
BrittAndreatta.comlynda.com