kfi innovation in learning presentation sept 2007

Eliminating Misinformation andDoubt in Learning

And Creating the Process of DiscoveryTo Achieve Mastery

Tim Adams

Chief Learning Officer & Co-Founder

Brian Webster

VP Client SolutionsVP Client Solutions

Copyright © 2007 Knowledge Factor, Inc.Copyright © 2007 Knowledge Factor, Inc. 22

Training DisconnectTraining Disconnect

People are being trained

but their performance doesn’t

always reflect it.


Disconnect Disconnect Food ManufacturerFood Manufacturer

• Implementing all the best practices and training

• Bringing uniform metrics to a $1B+ operation built through acquisitions

• Elated with business performance improvements - 70% efficiency!

• The cost of 30% inefficiency - $150M


Disconnect Disconnect Healthcare OrganizationsHealthcare Organizations

Number of preventable deaths 2003-2005 – 247,6621

1% of patients are injured in the hospital and 25% of these injuries result in death2

Medication errors harm 1.5M people each year3

Malpractice payments in 2004: $4.2B4

Preventable medication errors cost hospitals $3.5B per year5

1 HealthGrades, 2007; 2 Harvard medical study, 2006; 3 Nat’l Academy of Science, 2006; 4 Public Citizen, 2005; 5 Institute of Medicine, 2006


Why the disconnect?Why the disconnect?

Activity #1


The Impact of GuessingThe Impact of Guessing

People given credit for knowledge they don’t have

People given job responsibilities they are not prepared to handle

Management has false sense of security Hidden liabilities in organization Management doesn’t know that on-the-job

mistakes are a product of the training regimen


“Invisible errors” in the workplaceresulting from guessing correctly on true/false

and multiple choice tests are “common and potentially damaging.”

-Human Resource Guide Department of Labor study



Activity #2


Format Makes a DifferenceFormat Makes a Difference

Certain types of questions are prone to guesswork• T/F, Yes/No – 50/50 chance of getting

right, but not a measure of knowledge Content is not always aligned with

outcomes Content and assessment are not

always aligned with each other



Activity #3


The Value of a Second DimensionThe Value of a Second Dimension

More accurate predictor of behavior Focuses on knowledge quality Isolates knowledge states that can lead to

mistakes Provides moments of discovery Installs a new set of rules – focusing on

knowledge outcomes and not test scores! Confidence is the missing element in

the learning process


The Problem: Learning EffectivenessThe Problem: Learning Effectiveness

75%100%0%

A score only tells you how many questions someone answered correctly.


The Problem: Learning OutcomesThe Problem: Learning Outcomes

55%100%0%

A score can’t detect guessing or doubt.This may be what they really know.

75%


““The 55/15 RuleThe 55/15 Rule””

The average level of confidence and correctness for employees across all industries is 55%

15% of the knowledge employees have is actually confidently-held misinformation


Knowledge ProfileKnowledge Profile A Better Indicator of Potential PerformanceA Better Indicator of Potential Performance

Knowledge

Con

fid

en

ce Action

No Action

Learning Behavior Modeltm


What companies need is…What companies need is…

MASTERYMASTERY

Confidently-held, correct knowledge put into practice

How do you get there?


Demonstration


Case Study ResultsCase Study ResultsHealthcare OrganizationHealthcare Organization

Adverse outcomesMinimized with CBL

Positive outcomesMaximized with CBL

Misinformation extraordinarily highamong seasoned employees


Question: A worker in a science laboratory supervises and manages a department with 10 other full-time employees. She makes all the decisions about whether to hire, to fire or to promote employees within her department, but she is not senior enough to make big decisions about the direction of the company. What must the company consider when determining if the employees is an exempt executive?

Question: A worker in a science laboratory supervises and manages a department with 10 other full-time employees. She makes all the decisions about whether to hire, to fire or to promote employees within her department, but she is not senior enough to make big decisions about the direction of the company. What must the company consider when determining if the employees is an exempt executive?

Seminar Results onSeminar Results on Fair Labor Standards Act Fair Labor Standards Act Example of risk of Example of risk of not not knowingknowing

100 attendees in a conference65% were misinformedFine for not following this statute: $10,000Potential fines among attendees: $650,000

100 attendees in a conference65% were misinformedFine for not following this statute: $10,000Potential fines among attendees: $650,000


Tracking the Path to Mastery

… to eliminatepotential errors resulting from misinformation

CBL identifies knowledge quality of individuals …


Reporting AnalyticsReporting Analytics

GROUP PERFORMANCE

Item analysis by question allows an organization…

…to identify liabilities before they impact business objectives.


Confidence-Based Confidence-Based MeasurementMeasurement

Surgically isolates misinformation


Confidence-Based Learning Confidence-Based Learning DesignDesign

Metadata – • Module• Topic• Rich media

Question Answers

• Correct answer• Incorrect answer 1• Incorrect answer 2

Explanation Further training links

<Module>up

<Topic>

<Question Number>

<Rich Media File>

<Question Text>

<Correct Answer>

<Incorrect Answer 1>


<Explanation>

<Further Training Links>

Intro Text:

Link Text:

Link URL:


Evaluation tools• Test items,

assessments, certifications

• questions, correct answers, incorrect answers

Learning Objective

Learning Objective

Training Content

Training Content

TestDatabank

TestDatabank

Mapping Existing TrainingMapping Existing Training

Start with…


Supporting content• Case studies• Further info –

websites, PDFs, video, flash, etc.

Learning Objective

Learning Objective

Training Content

Training Content

TestDatabank

TestDatabank

Supporting Docs

Supporting Docs


Start with…



Supporting Docs

Supporting Docs

TestingTesting

Training Content

Training Content

Learning Objective

Learning Objective

<Module>

<Topic>

<Question Number>

<Rich Media File>

<Question Text>

<Correct Answer>



<Explanation>

<Further Training Links>

Intro Text:

Link Text:

Link URL:

Supporting Docs

Supporting Docs

TestDatabank

TestDatabank

Training Content

Training Content

Learning Objective

Learning Objective

Training Content

Training Content

Convert to CBL…Convert to CBL…


Importance of confidencein measuring knowledge

It is a predictor of behavior Provides cognitive moments of

discovery Provides accurate picture of

knowledge quality Provides the ability to correct

errors before Aligns learning with business Aligns learning with business

outcomesoutcomes


ADVANTAGESADVANTAGES

Confidence Based Learning is the next Learning imperative that:

• Proves acquisition of essential knowledge through an informal learning process

• Increases retention of essential knowledge• Improves the quality of the learning

experience and engages the learner to achieve mastery

• Enables training to focus on knowledge application and skills development

Moves focus of Learning from a score to achieving MASTERY!


SUMMARYSUMMARY

Moves focus of Learning from a score to achieving MASTERY!

Learning and assessment are both done in the same context – creating Contextually Relevant Learning and the opportunity for discovery!

Discovery process creates cognitive linkage and greater retention of material.

Engages learners in new ways Shifts the paradigm!


QuestionsQuestions&&

DiscussionDiscussion


THANK YOU!THANK YOU!

Tim Adams, [email protected]

Brian Webster, Vice [email protected]

www.KnowledgeFactor.com720-214-4874

kfi innovation in learning presentation sept 2007

Business

learning outcomes

knowledge outcomes

knowledge employees

correct knowledge

learning process

outcomes content

learning effectiveness

measure of knowledge