metrics: the good, the bad, and the ugly
DESCRIPTION
Metrics: The Good, The Bad, and The Ugly. Metrics: The Good, The Bad, and The Ugly. Can (John) Saygin, Ph.D. Assistant Vice President for Sponsored Project Administration Office of the Vice President for Research Professor, College of Engineering - PowerPoint PPT PresentationTRANSCRIPT
Can (John) Saygin, Ph.D.Assistant Vice President for Sponsored Project Administration
Office of the Vice President for ResearchProfessor, College of Engineering
Center for Advanced Manufacturing and Lean SystemsPhone: [email protected]
Metrics: The Good, The Bad, and The Ugly
Metrics: The Good, The Bad, and The Ugly
Outline
• Lean: What does it mean?• IOM 2013 Report• From Data to Metrics• Success Stories of Lean• We want to hear from you
LEAN
LEAN: SEE WASTE and ELIMINATE IT
Waste (Non-Value Added): Anything that adds cost, time, effort without adding value
Waiting
Over-ProductionDefects
Motion
Wastes
Inventory Transport
Under-Utilization of
Human Talent Over-
ProcessingDEFINE VALUE!!!
HOW DO YOU MEASURE WASTE?• Metric: a standard of measurement
• Performance Metric: Standards of measurement by which efficiency, performance, progress, or quality of a plan, process, or product can be assessed.
• Deviation |Actual| - |Target|
LEAN METRICS
Lean Metrics – the appropriate measurements and goals for the Lean Improvement activities
Commonly used Time-based Lean Metrics• Individual Cycle Time• Total Cycle Time• Queue Time• Total
• non-time based metrics, such as: cost, customer satisfaction, on-time delivery, and quality.
And many more…
TRAP… “averages”
• "If you can't describe what you are doing as a process, you don't know what you are doing."
PROCESS… FLOW
- W. Edwards Deming
DEFINE-MEASURE-ANALYZE-IMPROVE-CONTROL DMAIC must be a “continuous process”
• Define the system, the voice of the customer and their requirements, and the project goals, specifically.
• Measure key aspects of the current process and collect relevant data, including controllable and uncontrollable factors.
• Analyze the data to investigate and verify cause-and-effect relationships. Determine what the relationships are, and attempt to ensure that all factors have been considered. Seek out root cause of the defect under investigation.
• Improve the current process based upon data analysis using techniques: Future process
• Control the future state process to ensure that any deviations from target are corrected before they result in defects. Implement control systems and continuously monitor the process.
METRICS
• To successfully use lean metrics:– Standardize measurements
• Make sure the results are accurate and consistent– Metrics should be easy to collect
• Gather data where it is most useful– Make the Lean Metric Visual
• Make information accessible
• Goal: Predictable Output– Stable Output is more
important than spikes of outstanding performance
10
Success with Lean Metrics
Process Design & Improvement
Process Flow Validation
Process Flow Automation
Do the Right Thing Do it Right Do it Better
EFFECTIVENESS(Performance directly linked
to Desired Outcomes)
EFFICIENCY(Rate of Desired Outcomes)
WHAT HOW
Continuous Improvement: A Mindset
11
Operational Performance Metrics
Programmatic Performance Metrics
Eliminate Whenever Possible
12
Policies Functions ProcessesLaws , Regulations, Rulesat various levels
5 Why’s… Do not target the symptoms
Get to the Root-Cause
INSTITUTE OF MEDICINE
2013 Report
IOM Report (2013)
Mission, Vision, Goals
Complexity, Inconsistency, Cost
“Learning Health Care System”
Continuous Improvement
CTSA 2.0
Additional Complexity:
Network(12 sites in 2006 to
61 sites in 2013)
Lack of… Metrics
IOM 2013 Report – Page 8
Learning Healthcare System
Lean Enterprise
Are they different?
Where to Start?
Lean in IOM 2013 – Page 67
Evaluation -- Page 81
Evaluation Individual CTSAs (pp.82-83)
15 Metrics (Page 84)
June 2013 Meeting
Programmatic Performance MetricsOperational Performance Metrics
FROM DATA TO METRICS
“In God we trust… All others, bring data.”
W. Edwards Deming
Reference: "Data, Information, Knowledge, and Wisdom" by Gene Bellinger, Durval Castro, Anthony Mills
Data represents a fact or statement of event without relation to other things.
Information embodies the understanding of a relationship of some sort, possibly cause and effect.
Knowledge represents a pattern that connects and generally provides a high level of predictability as to what is described or what will happen next.
A) It is raining.B) The temperature dropped 15 degrees and then it started raining.C) If the humidity is very high and the temperature drops substantially the atmospheres is often unlikely to be able to hold the moisture so it rains.
The Big Picture
Reference: "Data, Information, Knowledge, and Wisdom" by Gene Bellinger, Durval Castro, Anthony Mills
Where to focus when determining metrics…
Actionable Information… Not just data
Tools: Data Collection, Analysis,…
Job by itself…
Myopic Nature: Single Metric!!!
Source:
Source -- http://www.juiceanalytics.com/writing/choosing-right-metric/
SUCCESS STORIES OF LEAN
An Example from UTSA Human Resources…
http://camls.utsa.edu/
HR at UTSA
Background• Human Resource (HR) managing
“On-Boarding Process” of new hires– Orientation, tax and insurance paperwork,
computer and email accounts, requests for keys, telephone line, parking, etc.
• Problems:– New hires often take 2 weeks to be truly “on-board”– HR team spends hours on missing data & error corrections
everyday– Lots of “waiting” among offices
You are hired!
Objectives:• Shorten new hire time• Improve work readiness• Increase compliance
Methodology:• Lean training for entire office• Value stream mapping
and implementation planning
The Improvement Project
HR at UTSA
Root Causes• Disconnected operations• Isolated resources• Ineffective (error prone)
paperwork process
Solutions• Redesigned workflow (new
“value stream map”)• Partners with OIT, ID Card
Office, Parking, etc.• Standardized web forms
The Findings
HR at UTSA
• New process provides integrated resources at the Orientation for New Employees
• New hires are ready on Day 2 versus 2 Weeks• 100% accuracy of I-9’s• $231,319 net savings
in 1st year
The Results
HR at UTSA
IT IS TIME TO HEAR FROM
YOU…
Think of processes or functions that you perform…
Write on a piece of paper:• The Good: You are comfortable with it. You do
it well.• The Bad: You are not comfortable with it.
However, you work so hard and you get it done.
• The Ugly: You are not comfortable with it. Your performance varies when doing it.
Do it yourself
Discuss as a team and compile your notes on a large sticky paper
Do it as a team…
Good Bad Ugly
Can (John) Saygin, Ph.D.Assistant Vice President for Sponsored Project Administration
Office of the Vice President for ResearchProfessor, College of Engineering
Center for Advanced Manufacturing and Lean SystemsPhone: [email protected]
Any questions, comments
