…and how to give it to themWhat CEOs Want
Patti Phillips, Ph.D.
* What CEOs WantROI Institute and ATD research show that the data CEOs receive are not demonstrating they are getting what they want out of their talent investment. (N=96)
MeasuresCurrently Measure
Should Measure Importance
Inputs and Indicators 94% 86% 6
Efficiency 78% 82% 7
Reaction 53% 22% 8
Learning 32% 28% 5
Application 11% 61% 4
Impact 8% 96% 1
ROI 4% 74% 2
Awards 40% 44% 3
* Analytics drives talent decisions.
HCA
Planning
Development Acquisition
Engagement
McKinsey & Company and The Conference Board report, False Summit: Why the Human Capital Function Still has Far to Go (2012), states that human capital analytics is the linchpin of human capital investment. It is the thread that runs through all things human capital.
1. Framework that includes measures important to all stakeholders.
2. Process model describing step-by-step how to capture, analyze, and report data.
3. Standards that ensure reliable implementation of the process model.
4. Reporting that answers the six critical questions senior executives ask.
5. Seamless integration that ensures we give CEOs what they routinely.
* Success with Analytics
Levels of Evaluation Measurement Focus
0 Inputs The input into the project in terms of scope, volume, efficiencies, costs Participants, Hours, Costs, Timing
1 Reaction Reaction to the project or program, including the perceived value
Relevance, Importance, Usefulness, Appropriateness, Intent to use, Motivation to take action
2 Learning Learning to use the content and materials, including the confidence to use what was learned
Skills, Knowledge, Capacity, Competencies, Confidence, Contacts
3 ApplicationUse of content and materials in the work environment, including progress with actual items and implementation
Extent of use, Task completion, Frequency of use, Actions completed, Success with use, Barriers to use, Enablers to use
4 ImpactThe consequences of the use of the content and materials expressed as business impact measures
Productivity, Revenue, Quality, Time, Efficiency, Customer Satisfaction, Employee Engagement
Typical Measures
5 ROI Comparison of monetary benefits from program to program costs
Benefit-Cost Ratio (BCR), ROI%, Payback Period
MeasurementLevels
Measurement Category
RecommendedTarget*
0 InputsNumber of programs, participants, costs, efficiencies
100%
1 ReactionReaction and satisfaction with experience, content, value of program
100%
2 LearningAcquisition of knowledge, skill, information, awareness, insights
80-90%
3 ApplicationUse of knowledge, skill, information, awareness, insights
30%
4 ImpactChanges in business impact measures such as output, quality, time, engagement
10-20%
5 ROICompares monetary benefits to cost of program, project, or initiative
5-10%
*Target percentage of programs is based on typical number and variety of HR initiatives.
* Measurement Targets
MeasurementLevels
Measurement Category
2006Status
2011Status
0 InputsNumber of programs, participants, costs, efficiencies
100% 100%
1 ReactionReaction and satisfaction with experience, content, value of program
92% 89%
2 LearningAcquisition of knowledge, skill, information, awareness, insights
48% 59%
3 ApplicationUse of knowledge, skill, information, awareness, insights
11% 34%
4 ImpactChanges in business impact measures such as output, quality, time, engagement
8% 21%
5 ROICompares monetary benefits to cost of program, project, or initiative
2% 11%
* 2011 Global Leadership Development Practices Survey Percent of Leadership Programs Measured at Each Level
N=232; large organizations with leadership development budgets averaging $3 million annually
8
Develop/ Review
Objective of Program
Develop EvaluationPlans and
Baseline Data
Collect Data During
ProgramImplementation
Collect DataAfter
ProgramImplementation
Evaluation Planning Data Collection
LEVEL 1: REACTION
LEVEL 2:LEARNING
LEVEL 3: APPLICATION
LEVEL 4: IMPACT
* ROI Methodology Process Model
Isolate the Effects
Convert Data To
MonetaryValue
Calculate the Return
OnInvestment
Generate Impact Study
Data Analysis Reporting
LEVEL 5: ROI
Intangible Benefits
Capture Costs of Program
Identify Intangible Measures
9
* Reporting
Chain of Impact
Reporting occurs at two levels: micro (investment) and macro (strategic). It includes quantitative, qualitative, financial, and non-financial data.
Reporting answers the six critical questions senior executive ask: So what? How much? How do you know? What is it worth? How can we improve? What are the risks?
It allows you to declare value, improve programs, influence decisions, and give CEOs what they want.
What is your story?
Inputs
Reaction
Learning
Application
Impact
ROI
Intangibles
* Seamless Integration Assess
Situation
Analyze Maturity
Integrate Results
Define Analytical Approach
Invest & Evaluate
Execute
Find Cause
Human Capital
Strategy
Fitz-enz, J., Phillips, P. P., and Ray, R. (2012). Human Capital Analytics: A Primer. The Conference Board. Research Report R-1500-12-RR, p. 23.
www.roiinstitute.net