key performance indicators (kpis) & metrics · key performance indicators (kpis): metrics vs....
TRANSCRIPT
Amit Chandak
Key Performance Indicators (KPIs) & Metrics
Director Senior Experience Owner
10/09/2018
2
Profile
Amit Chandak earned his LSSBB from USAA in 2016 and QAI Global
Institute in 2011. Amit is pursuing a Master Black Belt designation from
The Ohio State University and holds a Bachelor’s degree in Business
as well as an M.B.A.
Amit is currently working at USAA as Director of Experience Owners to
formulate and execute Process Excellence, Agile and Risk
Management strategies to deliver seamless members and employees
experiences.
Prior to USAA, Amit brings over 16 years Banking and Financial
Service Industry (BFSI) experience – worked at Apple as an IT Project
Manager, Process excellence & Business Reengineering consultant at
Accenture, TCS and Wipro while working with various global clients
across several business domains (Finance & Accounting, Investment
Banking, Supply Chain, Consumer Banking, Insurance, HR etc)
+1 210 240 1299 https://www.linkedin.com/in/amit-
chandak-62b0996/
Presented by Amit Chandak at ASQ San Antonio Section 1404
3
Agenda
• Difference Between KPIs vs. Metrics
• Types of Metrics
• Metric Test
• Challenges & Tips
• Identification Process & Tools
Presented by Amit Chandak at ASQ San Antonio Section 1404
4
Key Performance Indicators (KPIs): Metrics vs. KPIs
• Measures components of an activities within a
process
• Aligns to core steps in the business process
• Informs KPIs
• Measures the completion of a specific
business objective (Output) or desired end
state within a certain timeframe
• Includes thresholds of acceptability
• Informs key business decisions
Not all metrics are KPIs, but all KPIs are metrics
METRIC
Measurement of an
activity’s performance
KEY PERFORMANCE INDICATOR (KPI) Measure indicating a
process's performance
Key
Ch
ara
cte
risti
cs K
ey
Ch
ara
cte
risti
cs
Presented by Amit Chandak at ASQ San Antonio Section 1404
5
Levels of Metrics
METRIC
Measurement of an
activity’s performance
KEY PERFORMANCE INDICATOR (KPI) Measure indicating a
process's performance
• Identify and Measure metrics at each level of
Organization
• Ensure metrics are tied at each levels; e.g.
• Profitability at Executive Level
• Cost per application at Manager Level
• Productivity or Handle Time at Employee Level
Key
Ch
ara
cte
risti
cs
Metr
ic
Levels
Presented by Amit Chandak at ASQ San Antonio Section 1404
6
Performance Measurement: Manage Inputs, Report Outputs
Speed
Direction
Altitude
Vital Few Outputs
What does the pilot control?
Leading Metrics Lagging Metrics Predict
Inputs
Throttle
Rudder
Elevator
Presented by Amit Chandak at ASQ San Antonio Section 1404
7
Types of Metrics: Leading (Input & Process) and Lagging (Output)
Leading Metrics Predicts Performance (Output)
•Processing Time, Queue Time
•Exception Rate, Rework
•Backlog (Volume & Time)
•Utilization, Availability
•Average productivity per Agent
•Customer Complaints
•Skill – Cross Training
•Volume Received
•Timeliness (Delivery)
•NIGO
•Timeliness (Delivery)
•Accuracy
•Satisfaction (Customer & Emp)
•Cost of unit, Cost of Poor Quality
•Revenue
•Risk
On Time
Quality
Complete Step 1 Step 2 Step 4 Step 5
On Time
Quality
Cost
Satisfaction Step 3
Inputs Process Output
Leading Metrics Lagging Metrics
Inputs Metric Process Metric Output Metric
Presented by Amit Chandak at ASQ San Antonio Section 1404
8
The performance metrics chosen must be “S.M.A.R.T.”
▪ Is it clearly defined?
▪ Can it be easily generated without complex calculations? Specific S
▪ Is it easy to collect the data needed?
▪ Can it be compared to other data?
▪ Can the measurement be defined in an
unambiguous way?
Measurable M
▪ Can stakeholders influence it?
▪ Do we understand what drives the metric? Actionable A
▪ If the metric changed, would anyone care?
▪ Does it tell the whole story from a value stream perspective? Relevant R
▪ Can it be measured often enough to identify problems while
they are still relevant? Time -
bound T
SMART Test
Presented by Amit Chandak at ASQ San Antonio Section 1404
9
Specific S
Measurable M
Actionable A
Relevant R
Time -
bound T
Does Your Metric Qualifies SMART Test?
Poor measure
Timeliness: Number of wires
submitted by EOD SLO.
▪ All terms are clear
▪ Computing this is
simple addition
▪ The data is
straightforward to
collect
▪ The employee / unit is
in control on whether
they deliver on time
▪ Cannot tell whether
there were any wires
that were late
▪ The number can be
calculated at any time
SMART measure
Timeliness: % of wires
submitted by EOD SLO.
▪ No change
▪ No change
▪ No change
▪ The score incorporates
the total number of
wires
▪ No change
Presented by Amit Chandak at ASQ San Antonio Section 1404
10
Specific S
Measurable M
Actionable A
Relevant R
Time -
bound T
Poor measure
Quality: Number of errors in
customer submissions per year
▪ Errors are clear
▪ The data is straightforward
to collect
▪ Does not provide context
(e.g. if 3 fail is that a lot or a
little)
▪ Associates cannot prevent
customers from submitting
error filled materials
▪ One year is required before
a poorly performing
process for detection will
be noticed
SMART measure
Quality: % of errors that are not
caught in the initial screen
▪ No change
▪ No change
▪ The % indicates the volume
of undetected to detected
errors
▪ Associates are able to
control the quality of their
screening
▪ The number can be
computed as any point
Does Your Metric Qualifies SMART Test?
Presented by Amit Chandak at ASQ San Antonio Section 1404
11
Watermelon Effect
Presented by Amit Chandak at ASQ San Antonio Section 1404
12
Process Integration – Backward and Forward
Marketing Acquisition Fulfillment Service
Design Raw Materials Production Distribution
Integrate your KPIs to get optimal business results
Presented by Amit Chandak at ASQ San Antonio Section 1404
13
Best Practices for Identifying Metrics and KPIs
Define End to End Process
1
Brainstorm Metrics 2
Distinguish KPIs vs. Metrics 3
Measure & Specification Limits 4
•Identify Metrics (from list created in Step 2) and
document additional details to enable data
collection
•Baseline information and specification limits
•Identify most important indicators of
successful Process performance
•Identify Input-Process-Output metrics
•Brainstorm operational metrics that will help
monitor and manage the process
•Develop SIPOC or High-level process maps
•Identify customer (internal/external) and
suppliers
Identify and Remediate Gaps 5
•Identify measurement gaps in
current state and remediate
accordingly
Presented by Amit Chandak at ASQ San Antonio Section 1404
Data Collection Plan
14
Process Name: Pizza Delivery Process Owner (PO): Pizza Store Manager
LoB: Suppliers Inputs Input Requirements Process Outputs Customer Requirements Customers
Who Provides the
inputs?
Materials, Information,
Ideas, Labor
How does the Process Owner
want the inputs? Starts with: Hunger
Documents, Information,
Services, Decisions
How does the Customer
want the outputs?
Who Uses or
Benefits from the
Output?
Store Manager (PO)
Store Manager (PO)
Hungry customer
Store Manager (PO)
Menu options
Order venue (phone, digital,
,com)
Delivery information (name,
address)
Stated delivery Service Level
Agreement (SLA= 45 minute
delivery)
Full menu options
Order venue (phone, digital, ,com)
Accurate information (name, address)
Time order is entered
Order Pizza
Documented order
Order is in queue
Estimated delivery time
Delivery information
Order recorded
Order is in queue
Accurate delivery lead time
estimate
Accurate delivery information
Baker
Baker/Hungry customer
Hungry customer
Driver (Task Owner)
Order Taker (Activity Owner)
Order Taker (Activity Owner)
3rd Party supplier
Pizza store Manager (PO) /
Baker (Activity Owner)
Documented order
Order in queue
Ingredients
Standard operation procedures
(SOP)
Accurate order
First-in-first-out (FIFO) order
Fresh ingredients available
Ingredients added in proper proportions
Assemble Pizza
Assembled pizza Pizza assembled accurately per
order (100% of the time)
Hungry customer
Baker (Activity Owner)
Store Manager (PO)
Baker (Activity Owner)
Assembled pizza
Pizza oven
Time in oven
Assembled pizza accurate
Pizza oven at baking temperature Bake Pizza Baked pizza ("done") Thoroughly baked pizza / not
burnt
Pizza visually appealing
Hungry customer
Scheduler (Activity Owner)
Scheduler (Activity Owner)
Store Manager (PO)
Store Manager (PO)
Estimated delivery time
Time in delivery queue
Volume of deliveries in queue
Thermal carrier
Delivery route
Delivery driver
Delivery information and vehicle
On time delivery
Minimize queue time
Minimize queue volume
Insulates content (keeps pizza hot)
Efficient route
Available delivery driver
Delivery staff
Reliable delivery vehicle
Deliver Pizza
Pizza delivered
On time delivery (45 minutes
from order)
Hot pizza
Correct toppings
Hungry customer
Ends with: Eat Pizza
Ends with: Eat Pizza
Step 1 - Define End to End Process using Gold SIPOC
15
Steps 2 - Data Collection Plan
Date
Captured
Associated
CTQ
Category
Metric Name Operational Definition Math Formula
Unit of
Measurement
Description of Business
Insight / Objective
XX/XX/XX CTQ from the VOC
Is the metric related
to Time, Cost,
Quality, or Member
Experience?
What should the metric
be called? Should be
easy to understand.
How would you describe the metric so there is no
ambiguity on what you are intending to measure?
How would you calculate the
measure? For instance, would it be
a sum of all products, a ratio, the
difference between two values, etc.
e.g., #, $ , % What would this metric enable you
to do? What will it be used for?
Presented by Amit Chandak at ASQ San Antonio Section 1404
Step 2. Brainstorm operational metrics that will help monitor and manage the process, and identify KPIs that depicts performance.
16
Steps 3 - Data Collection Plan
KPI? Other Conditions / Filters to record
Included in a
report /
dashboard?
Name of report /
dashboard
Current Data Source &
Location
How is the data
collected?
Who collects
the data?
What
Frequency?
Yes / No
What other types of filters would be needed to
make this data more meaningful? For instance,
would you need to be able to filter by product or
by employee?
Is the measure
being captured in a
report/dashboard
today?
If the measure is
captured in a report,
please list the name of the
report .
Where is the data being
stored?
Is the data being
captured manually
or automatically?
List the name of
the individual or
system.
What frequency is
the data collected
?
Presented by Amit Chandak at ASQ San Antonio Section 1404
Step 3. Identify Metrics (from list created in Step 2) and document additional details to enable data collection
17
Step 4 - Data Collection Plan
Industry
Benchmark Baseline
Target
State Lower Spec Limit
Upper Spec
Limit Data Available
Notes on Data
Availability What is the current
performance of the
metric in the industry?
What is the current
performance of the metric?
What is the target state?
(Should be informed by
VOM / VOB).
What is the lower limit?
What is the upper limit?
(There might not be an
upper limit)
Yes, No, Partially Available Note as GAP if data not
available
Presented by Amit Chandak at ASQ San Antonio Section 1404
Step 4. Once data is gathered, enter in baseline information and specification limits.
18
CoSA: Process Owner: Pizza Store Manager
Process Name: Pizza Delivery Last Update:
3X3 Matrix
Y Y
(Output, Lagging, Reactive)
X1 (Input, Leading, Predictive)
X2 (Input, Leading, Predictive)
X3 (Input, Leading, Predictive)
Y1 Quality: Pizza toppings
correct Accurate order taken Pizza assembled per order Ingredients available
Y1 Quality: Hot pizza Delivery Lead Time Thermal carrier efficiency Time in delivery queue (wait/delay
time after baking)
Y2 Speed: On Time Delivery
(45 Minutes) Order queue time Time in delivery queue Delivery route efficiency
Y3 Cost: Cost per pizza Defect rate Delivery efficiency (routing) Missed delivery commitment
3 x 3 Matrix
•Vital few vs Trivial many – Defining and monitoring the vital few is essential so team is
focused
•Identify 3 most important Lagging Metrics (KPIs) and 3 most influential Leading Metrics
•Proactively control the inputs that significantly impacts the output metric
Presented by Amit Chandak at ASQ San Antonio Section 1404
19
Best Practices for Classifying Metrics
To Be Retired To Be Migrated To Be Developed
Associated Risk
Imp
act
to C
usto
mer
Associated Risk
Imp
act
to C
usto
mer
Next Steps:
• Document as-is process for tracking
and reporting the metrics/KPIs
• Create streamlined process for
reporting metrics/KPIs
Next Steps:
• Meet to determine gaps in obtaining
metrics/KPIs not currently in
production
• Develop interim state method
for obtaining metric/KPI
Next Steps:
• Communicate decision to retire
metrics/KPIs to stakeholders who are
responsible for their production
Gap Remediation Strategies – (Step 5 of 5)
Presented by Amit Chandak at ASQ San Antonio Section 1404
20
Continue The Cycle
What get measured, gets reviewed and improved!
Identify
Measure
Improve
Sustain & Recycle
Presented by Amit Chandak at ASQ San Antonio Section 1404
21
Presented by Amit Chandak at ASQ San Antonio Section 1404