transform organisational culture - defining, communicating and measuring core values

By

Raed S. Haddad Managing Director ESI International

Determining Value of Investment— A pragmatic ROI for Measuring Learning

Investments

© 2014 - All information in this document is copyright protected and the property of ESI International, Inc.

© 2014, All rights reserves

The leading global provider of PERFORMANCE IMPROVEMENT programs in:

What We Do 31

YEARS OF EXPERIENCE

PARENT COMPANY

The world's leading private equity firm

focused on media, communications,

education and information investments

Project &

Program

Management

IT Project

Management

Agile Contracting Business

Analysis

Business

Skills

Portfolio

Management

PARTNERS



Where We Are

US Offices • Arlington/Washington DC • Chicago • New York • San Francisco

ESI Partners • Japan • Poland

Global Offices • Australia • Brazil • Canada • China • Germany

• India • New Zealand • Singapore • South Africa • Spain • Sweden

• United Arab Emirates • United Kingdom



Among Our Clients

My Commitment

1. Mismanaging change

2. Not listening to customers

3. Tolerating low performance

4. Not facing reality

Why CEO’s Get Fired?

6

Turning Strategy into Results

Source: Leadership IQ


Agenda

Current State of Affairs

Adding Value Putting it Together


Current State of Practice



L&D—Focus on Productivity & Impact

What do CxO’s think?

What does Middle Management think?

How much “Scrap Learning” exists?


ESI’s Global Learning

Application Survey


Do you have a Formal Process to Ensure OTJ Application?

___%

___%

67 33


How Much (%) is Applied?

BUT….


How Do You Know That?


Agenda




Who determines value?

Target Audience

YOU


Thinking about a NEW Paradigm

Build on Existing Data Collection Points

ROI VOI


What is Your Definition of Value

Improve performance: • Improve business operations

performance end-to-end across the enterprise

• Increase customer and employee satisfaction

Minimise risk: • Ensure continuity of internal

business operations, while minimising exposure to risk factors

Maximise return: • Improve business results;

grow revenue and earnings, cash flow, and reduced cost of operations

Increase agility: • Enable the business organisation and

operations to adapt to changing business needs


Value—in the Language of Business

– Effective and profitable management of complex projects

– Global consistency and capability

– Effective use of resources

– Higher customer satisfaction

– Increased employee satisfaction

– Significant competitive advantage


Agenda




21

Make the Case for Change

Change Identification Change Engagement Change Implementation

AS IS TO BE

Roles

Ind

ivid

ual

Co

ntr

ibu

tors

Div

isio

n

Co

rpo

rati

on

Man

ager

s E

xecu

tive

s S

r. M

anag

ers

Discover AS-IS Individual

Skill Development

Management Development

Executive Development

Discover AS-IS Managers

Discover AS-IS Organisation

Articulating the Problem

Articulating the Vision

Alignment to Strategy

Reinforcement Interventions

Coaching and Reinforcement WS

Performance Support Systems


Address the Manager Problem

New Program

Maturing Program Established Program

Ind

ivid

ua

l &

Org

an

isa

tio

na

l M

atu

rity

Behavior Change Over Time

Improved

Business

Success


Measure

Increasing quality

Increasing productivity

Increasing employee engagement

Decreasing costs

Increasing revenue

Increasing customer satisfaction

Decreasing cycle time

Decreasing risk

Increasing effective communication


From Smile Sheets to . . .

Level 1 Reaction

Did you enjoy class?

Facility & Food!


Smart Sheets. . .


Game Changing Conversation Results and Metrics for ESI Learning Investments across 6 Levels of Measurement

1. Quality – How satisfied were students with the quality of the ESI learning experience, as measured by feedback on instructor, courseware and environment?

2. Learning Effectiveness – To what extent did students report learning new knowledge and skills?

3. Job Impact – What did students report about predicted and then reported success in applying ESI knowledge and skills on the job?

4. Business Results – To what extent did students say their performance would improve as a result of ESI training? What specific business results did they say they would impact following training?

5. Return on Investment – Was the training worthwhile for them and their employer? And what evidence exists of financial ROI for the learning investment

6. Net Promoter Score


The Lens Through Which we Can Tell This Story

– Entire Portfolio (all courses, all clients, globally)

– Individual Client

– Region

– Modality (ILT and vILT)

– Curriculum

– Course or Course Groups (i.e. a certification path)

– Individual Class session

– Instructor


Tools We Use To Support the Conversation

Standard Reports

Insights from ESI

– Before the course—setting expectations

– Immediately after

– Maximising response rates

– 60-90 days follow up

– The managers involvement

– The case against predictive analytics

– Client Program Review Deck


Online course end evaluations: a) Level 1 & 2 b) Predictive Level 3 c) Predictive Business Impact

Online follow-up 60-90 days—with participant and with their manager about b & c above

Setting Expectations


Make Decisions With Impact Data

6.38

6.26

6.25 6.18

5.50

5.36

5.33

Requirements Management: A Key to Project Success Project Leadership, Management and Communications Managing IT Projects Taking Charge of Organisational Change Budget and Financial Management Project Management Essentials PMP® Exam Preparation

Average – 5.81


Job Impact

Training impact on performance improvement, adjusted for bias:

18% training impact on performance improvement

84.4% x

59.1% x

56.7% x

65% =

Item

Given all factors, including this training, estimate how much your job performance related to the course subject matter will improve.

Based on your response to the prior question, estimate how much of the improvement will be a Direct result of this training.

What percent of your total work time requires the knowledge/skills presented in this training?

Bias, confidence, conservatism

Amount

84.4%

56.7%

59.1%

65%


Business Results

Increasing quality

Post

Event Follow Up

41.37% 42.21%

52.69% 56.18%

21.80% 23.71%

65% 65%

14% 15%

Decreasing costs

Post

Event

Follow

Up

32.22% 32.17%

46.80% 53.70%

15.08% 17.28%

65% 65%

10% 11%

Increasing

productivity

Post

Event

Follow

Up

38.94% 40.78%

51.90% 51.25%

20.21% 20.90%

65% 65%

13% 14%

Total percent improvement, including

training

Training's contribution to improved

business result

Percent improvement due to training

Adjustment factor for confidence in

estimations

Adjusted percent improvement due to

training


Other Metrics

1. Learning Effectiveness

2. VOI

3. Support Tools

Support Tools

1+ 2+ 3+ 4+ 5+ 6+ 7+

Post-training manager discussion 15.25% 6.78% 10.17% 20.34% 24.29% 15.25% 7.91%

Adequate resources 3.72% 7.98% 9.57% 24.47% 25.00% 20.21% 9.04%

Pre-training manager expectations 15.25% 6.78% 10.17% 20.34% 24.29% 15.25% 7.91%

On the job materials 15.25% 6.78% 10.17% 20.34% 24.29% 15.25% 7.91%

After training, my manager and I will discuss how I will use the learning on my job.

I will be provided adequate resources (time, money, equipment) to successfully apply this training on my job.

My manager and I set expectations for this learning prior to attending this training.

The participant materials (manual, presentation handouts, job aids, etc.) will be useful on the job.


Client Program Review Presentation


Client Results Flyer


What Did Our Stakeholders Say? “I am GA-GA over this stuff! I will stalk you for the data!”

Rockwell Collins

“This will streamline the type of information I have to share with our sponsors.” MTM reports will save me so much time!”

KBR

“We have to be thinking about this type of learning evaluation data for all of our training.”

McCormick

“MTM data will help me have a management discussion about the value of the classes.” General Dynamics

“MTM Reports provided us with evidence that supported recommendations to improve the program following the very first delivery.”

Hospira

“In the absence of proof, let evidence be

enough.”


38


It Does Not End Here For further information, contact:

Raed S. Haddad

Managing Director

+65 6505-2047

[email protected]

mailto:[email protected]



transform organisational culture - defining, communicating and measuring core values

Business

rights reserves ldfocus

rights reserves valuein

business results

business needs

business organisation

property of esi international

language of business

value of investment