“simulation scorecards as drivers for calculating roi in addition to feedback” by ken spero-...
DESCRIPTION
Ken Spero speaks about “Simulation Scorecards as Drivers for Calculating ROI in Addition to Feedback” at Serious Play Conference 2012 ABSTRACT: One of the biggest challenges facing learning professionals in organizations in every sector is justifying the cost of training. With the economy the way it has been, any level of spending is challenged with respect to the expected returns and without the justification, budgets are scarce. Much of the training that organizations need to achieve the necessary change in employees to affect performance relates to soft skills which are difficult to assess or measure under the best of circumstances. This is compounded by the fact that few training initiatives happen in a vacuum and it is therefore difficult to isolate the impact of the training versus other aspects of the initiative. For many years, Simulations have been a key vehicle for capturing and deploying Experience in a ‘risk free’ manner that approximates the ‘school of hard knocks’ but without the associated bruising. Articulating a Scorecard is a key step in Experience Design and Simulation development. Experience Design is built on the premise of being able to articulate and then operationalize the experience you want a participant to have. One of the initial key steps in articulating experience is to understand how success, or failure, is measured in that experience. By breaking down learning objectives to how they can be observed helps to make the intangible measurable. Once it is measurable, it is not so difficult to make it quantifiable.TRANSCRIPT
Presented by:
Ken Spero
Simulation Scorecards as Drivers for Calculating ROI
in Addition to Feedback
Serious Play Conference
August 22, 2012
Today’s Session Outline
In this session we will address the following issues relating to Scorecards and ROI:
1. Understanding the value of Simulation2. The role of the Scorecard in Sim 3. Experience Design Methodology4. Balanced Scorecard5. Building the Business Case – ROI6. Look at and Play a Sim (if we have time)
Why use Simulation?
Experiencing Best Practices Enables Critical Thinking
Why use Simulations?
Experience is the best teacher Sims provide experience (emotional engagement
with environment) Sims expand the evocable experience base, they
become part of your experience portfolio / “gut”. Sims consequate Mindlessness and encourage
Mindfulness Sims provides an opportunity for participants to learn
from failure, to Fail Forward Sims enable time acceleration to feel consequences Sim leverages the power of storytelling (context)
When Sim
Experience Design: Authoring Simulations (Introducing the Assignment)
1. Establish the Narrative (something that can be
re-used – Anchor Story)
a) Clearly define a problem / Target Learning Objective
b) Articulate the desired experience and outcome
c) Timeline and flow
d) Create Measurable Scorecard: Goals and Boundaries
e) Establish Context: Environment / Storyline / Cast
f) Start with ‘Golden Path’
Experience Design: Authoring Simulations (Cont’d)
2. Articulate and Prioritize Key Learning Objectives
3. Write a Scene from narrative that reflects each of the
selected Learning Objectives
4. Articulate Alternatives/Choices/Feedback
5. Lay out scenes in the flow of the narrative
6. Determine Memorable and Realistic Consequences
7. Add intros and outros
8. Score Choices – Link to Objectives in a Measurable way
Simulation Scorecards
Scorecards are a very powerful component for both:
Feedback
Authoring
Especially when there are tradeoffs between the different objectives or between different stakeholders
Example – Project Leadership
Interpersonal and organizational aspects of projects Focus of Sim on both Team and other Stakeholders Team Scorecard Organizational Scorecard
– Focus - Customer Satisfaction– Motivation - Management Satisfaction– Skill Development - Perceived Product Quality– Buy-in/Commitment - Team Total– Productivity - Project Total– Innovation - GANTT Chart (Scheduling)– Communication - Budget– Leadership
The Balanced Scorecard Methodology
Start with a business problem that needs to be addressed – this is the underlying reason driving the need for the training
If we are able to solve this problem, what will I see change or benefit or how does the problem we are trying to fix manifest:
– From a Financial perspective– With respect to my Customers and their Satisfaction– With our Internal Management Processes– With our ability to Innovate and Grow
Linking the Scorecard and ROI
Back to the business problem …
When it comes to ROI it is helpful to consider where the potential returns will come from with respect to solving the Business Problem:
– Quantified Business Benefits– Avoided Costs and Reduced Risk– Cost Changes
Developing the business case
Determine how and where the issues manifest at their most compelling level by examining the:Benefit – Beneficiary Matrix
– Benefit - What is the nature of the problem: improved – Efficiency, Effectiveness or Innovation – Faster, Better, New
– Beneficiary – Where is this most compelling place to measure: Enterprise, Group or Individual
The Balanced Scorecard Methodology
Start with a business problem that needs to be addressed – this is the underlying reason driving the need for the training
If we are able to solve this problem, what will I see change or benefit or how does the problem we are trying to fix manifest:
– From a Financial perspective– With respect to my Customers and their Satisfaction– With our Internal Management Processes– With our ability to Innovate and Grow
Break Down Balanced Scorecard to Measurable Components
Identify observable behaviors that demonstrate the scorecard items – these might be specific competencies relating to the business problem or
Define Units of MeasureIdentify units of Measure - from Benefit/Beneficiary MatrixIndividual Level
– Productivity – units/day– Sales productivity – won/loss ratio
Group Level– Sales org productivity – cost of sales– Workgroup productivity - # of staff– Improved Customer Service – number of complaints
Enterprise Level– Improved Product Quality – defect rate– New product capability – volume of new products/year– New market entry – volume of new markets/year
Do the Math Use the units of measure to quantify benefits and/or
avoided costs and think about it over time Compile the expected cost of development and
deployment Establish a baseline cost – ‘what if we did nothing…’ Also determine what you are competing against (i.e. an
alternative approach or some other usage of the money)
Find someone to help with the calculations, if you need it, based on the expected complexity of the numbers and metrics
Useful Links:
www.seriousplayconference.comwww.seriousgamesdirectory.comwww.seriousgamesassociation.com
Contact: