kpi & dashboard in quality operation - smart-qc · pdf filecontinuous improvement...

Continuous Improvement Platforms

Boosting Operating Performance

KPI & Dashboard

in Quality Operation

www.cresultsconsulting.com

cResults / ERD confidential 1

Resource Planning, Scheduling,

Cost of Quality and Efficiency

Management for QC Laboratories

www.smart-qc.com

Batch Record Release Management

and Overall QA Scheduling &

Efficiency Management

www.cmanageefficiency.com

By cResults an IPS Affiliate

WEBINAR AGENDA

Introduction - Today's Drivers, and the need for improved

visibility

Key Performance Indicators Goals and Objectives

KPI Overview and Examples / What Data Need to Be Collected /

How to calculate an more


How to calculate an more

Demo of cME / Smart-QC as a platform to capture all related QA

& QC data requirements & Reports / KPI / Dashboard

Case Study

Summary and Q&A

Today’s Business Drivers

Release Time Reduction

On-Time Delivery &Delivery Consistency(customer service)

Overall Cycle Time


Efficiency

Variability

Overall Cycle Time

& Reduce Cost

Compliance

XX … these challenges have been

recognized by corporate leaders

as imperative issues that need

to be addressed.

Definition of KPI

In today’s Webinar we will focus on several KPIs, and how to define,


In today’s Webinar we will focus on several KPIs, and how to define,

measure, and report them:

Batch Record Management

QA Professionals Efficiency (QA Operation)

Service level – Focused on cycle time and on-time delivery

RFT Documentation - QC

Non Traditional Compliance KPI

�

��

��

�

�

# Focus Area Key Performance Indicators (KPI) QA Operation

KPI Examples QC Operation

1 Compliance • Investigation Open above 30 days; New vs.

Closed; Investigation TAT; CAPA Effectiveness

• Regulatory Commitments

• Complaints

• Activities based KPI (e.g. daily audits, clearances, swabs) & Failures / observations while performing these activities.

• OOS Rate / Investigations

• OOS Closure Turn-around Time (TAT)

• Stability Test On Time %• Right First Time Documentation

• Error per analyst

• Errors Type Distribution

2 Right First Time

• Right First Time (by area / product)• RFT by QA tech / by Owner• Breakdown of documentation errors

• Re-test Rate• Pending Information

3 Financial and Efficiency

• Overtime Rate

• Budget Variance

• Reviewer’s efficiency

• Overtime Rate

• Budget Variance

• Analyst Earned Hr. vs. Actual Hrs.


• Reviewer’s efficiency• QA Techs Earned Hr. vs. Actual Hrs.• QA Tech Workload vs. QA Tech Availability %

• % of Time Spent On QA VA vs. Non-QA Activities

• Analyst Earned Hr. vs. Actual Hrs.• Analyst weekly efficiency %• Cost per sample / product / VS• Efficiency losses Pareto• Instrument utilization (Actual/STD)

4 Customer Service

• Release cycle time• Schedule Adherence• Reviewer cycle time / Mfg./Pkg. response time

for documentation errors• Investigations / CAPA closure Turn Around Time

• Sample turn around time / Cycle Time

• On Time Delivery / Schedule Adherence

• Backlog

5 People / Training

• Cross Training

• Turn-over / Hiring

• Safety

• Cross Training• Turn-over / Hiring

• Safety

6 New Product Introduction

• Quality of the Batch Record • Re-test Rate / Turn-around time

QC Dashboard Example

Throughput


On-Time Delivery

Cycle TimeEfficiency

cME – QA Dashboard Example

Efficiency


Errors Recorded RFT / Errors by Area

Throughput Cycle Time

cME – QA Dashboard Example


KPI – measure progress toward Operation Excellence QC and QA

Meaningful KPI –measuring the progress

An effective labs’ structurethat support the business goals

Efficient Layout / Material Flow and Work Method / Make or Buy

Meaningful KPI –measuring the

progress

Reviewers performance,guidelines and harmonization

Clear roles and responsibilities(Mfg./Pkg./QA/QC)

QA Release Optimization QC Efficiency Improvement


and Work Method / Make or Buy

Effective coordination & alignmentwith Mfg. / Planning / purchasing

Automation and adequate resources level(Instruments and Analysts)

Systems: LIMS / Data Acquisition / Planning / Daily Scheduling / Test Allocation / Training

Detailed Labs’ standards as a foundation for effective planning and Scheduling of the Labs’ resources

(Mfg./Pkg./QA/QC)

Value added vs. Non Value added reviews (Stage / Full Review)

Right First Time documentation –awareness, standardization, training, and

accountability

Investigation Closure improvement, early detection and determination, effective root

cause and CAPA

Batch Record design Streamlining (Layout, Clarity, Simplicity, and Standardization)

Batch Record KPI needs

What is the Right First Time (by area / product)?

What is the QA review cycle time?

What is the Mfg./Pkg. response time to correct documentation errors?

What are the main delays / pending issues causing BR delays?

What is the impact of investigations / CAPA closure on the BR

��


What is the impact of investigations / CAPA closure on the BR release time?

What is my reviewer’s efficiency?

What is the RFT by QA tech, and what leads to the variability between QA techs?

What are the causes for time losses for my reviewers?

What are the value added reviews / stages throughout the release process?

What is the breakdown of the documentation errors?

Print error logPrint error log

1st Review 2nd Review

1st Mfg./Pkg.

Response time

2nd Mfg./Pkg.

Response time

Improve Batch Record release process

?

��


Insert Time

Assigned Date

Review Start

Review End

Mfg./Pkg.

Insert Time

Assigned Date

Review Start

Review End

Pass 2nd

(1) Cycle Time

(2) RFT

(3) Efficiency

Pending

?

Pass 1st

Meaningful KPI��


Earned Hours = Σ All Activities Performed * Standard time to perform these activities

Available Hours = Available direct staffing hrs – Σ All the planned

Efficiency in QA��


hrs – Σ All the planned unavailable activities (i.e., meetings, training)

During A Given shift a QA Tech performed the following:

� Release 2 BR – Type A (2 * 1.5 Hr.).

� Fail BR – Type B after review (45 min. for Review and 15 min. if

errors found).

� Performed an Audit (1 Hr.).

� Participant in 1 Hr. EHS training.

Efficiency Calc. Example (QA)��


Based on the system standards the overall Earned Hrs. were:

� Earned Hours = Σ 3 hrs. + (45min+15min) + 1 Hr. = 5 Hrs.

� The shift duration is 8 Hrs.

� Available Hours = 8 hrs. – 1 hr. = 7 hrs.

QA Tech Efficiency = (5 / 7) * 100% = 71.4%

Tests / Audits

Tests & Audit [Earned Hrs] / (Weekly Schedule – Unavail – Non Tests) * 100% = Weekly Efficiency

Efficiency Calc. Example (QC)��

cResults / ERD confidential15

Unavail Events

Weekly Schedule

Non Tests

�� Unavailable time

cResults / ERD confidential

Available Hours = Available direct staffing hrs – Σ All the planned unavailable activities (i.e.,

meetings, training)

Overall Mfg. / Packaging Execution

Cycle Time Measured from Batch Record Issuance to BR Release

Main Components:

Batch Record Review Time

QA tech workload �� Efficiency �� Affect Cycle Time

Cycle Time / Turn Around Time (QA)��


QA tech workload �� Efficiency �� Affect Cycle Time

BR Delays:

Investigation / CAPA – Turn Around Time

QC Sample Release

Mfg. Turn Around Time (BR Errors)

Other …

S0

E4

S1

*E1

*S2

E2

S3

E3

S4

Arrival of the Sample

Start of Batch 1 out of N

Complete the Run of the instrument

Start Post Run Calc.

End Post Run

Start Audit

End Audit

Start Approval

End Approval

Q1

Q2

Q3

Q4

Qu

eu

e T

ime

P1

P2

P3

P4

P5

Cycle

Tim

e / T

urn

Aro

un

d T

ime (Q

C)

��

cR

es

ults

/ ER

D c

on

fide

ntia

l18

Arrival of the Sample

Start of Batch 1 out of N

Complete the Run of the instrument

Start Post Run Calc.

End Post Run

Start Audit

End Audit

Start Approval

End Approval

Pro

cess / T

est

Sample ID

Product Id

Product Desc

Batch ID

Test ID

Sub Test ID

Test Desc

Lims Unique ID

Insert Date

Start CT

Batch Start

Cycle Time Report from Smart-QC

��


Batch Finish

Testing CT

Audit Start

Audit End

Audit CT

Review Start

Review End

Review CT

Comments

• Audit Report – Documentation Review in QC

– RFT per Week

– RFT per Week Chart

– RFT BY SUPERVISOR

– RFT BY Analyst

RFT Documentation in QC (From Smart-QC)��


– RFT BY Analyst

– Errors Type Distribution

– Errors BY Owner

20

��

Errors Type Distribution

RFT Documentation in QC (From Smart-QC)


• More Common: – Investigation Open above 30 days; New vs. Closed

– Regulatory Commitments

– Complaints

– More …

Non Traditional Compliance KPI�


DEMO – Dashboard and KPI Examples


Case Study – Par PharmaceuticalA Full Article could be found on cResults Website –

www.cresultsconsulting.com


StepStepStepStep----1111:::: cME provides an

overview on error reasonsbreakdown for all areas. Thisgives us the first indication ofwhere the issues could befound

StepStepStepStep----2222:::: Takes us a step further

to find out by month which ofthe error code are the dominantones

StepStepStepStep----3333:::: Identify if the errors are

contributed by a specific area.In this example, Pharmacy was

Step-1

Step-2

StepStepStepStep----5555:::: Provide the right first time metric by each

individual QA reviewer. Once a substantialdifference is detected, it is pointing the team toimprove the standardization process among thereviewers.

StepStepStepStep----6666:::: In this step we will look closer on the type

of errors made. While in previous steps, the mainfocus was on the type of errors, in this step, we arefocusing on the details level which can lead us to aspecific page that may cause issues, a specificinformation that is often missed etc. This can leadto changes of the actual batch record design alongwith some targeted training for the production

Step-5

Step-6

RFT by QA Reviewers

Detailed Comments by BR

Leveraging KPI to Reduce Batch Record Documentation Errors

In this example, Pharmacy wasthe lead area in terms of errorsgeneration, which greatly helpsthe team to focus on this area toget a quick win.

StepStepStepStep----4444:::: Focus on a pareto – find

out which of the errors are themost dominant in a specificmonth. As in step-1 we werefocusing on the overall errors,in step-4 we are looking if atrend is detected for a monthspecific details. This is anotherstep in identifying if the errorsare related to a specific areaor/and a few specific errorcodes.

Step-3

Step-4

team.

StepStepStepStep----7777:::: cME provides the view of the same data

previously analyzed from the productionownership. In other words, once an error isrecorded, in addition to the area, the error code,the QA check that are being recorded, theproduction owner that could be a group leader, ora supervisor is recorded as well. This allows theteam to identify some specific issues related to anindividual that may need additional training orsome clarifications.

StepStepStepStep----8888:::: Finally, cME provides a robust cycle time

and efficiency data that could show us not just theerrors but also the impact of QC samples,investigation, pending for errors response, etc. sothe team could once again focus on the mostcritical area / individual / events that lead to cycletime losses and overall deficiencies.

Step-7

Step-8

3 Executive Drive 2nd Floor, Somerset, NJ 08873 P: 732-748-1990; F: 732-748-1993www.cmanageefficiency.com

Case Study – KPI To Measure Progress

� Errors in batch records were reduced in the month of August

by 29.7%

� Response time of Manufacturing to correct errors was

The case study outlines the results observed after 3 months of cME implementation at Par Pharmaceutical a

leading generic companies:


� Response time of Manufacturing to correct errors was

reduced by 18.7%

� Reviewer efficiency is up over 10%

� With cME information, the number of reviews are expected to

be reduced by 50%

� Many traditional QA activities are being challenged via

cME’s visibility of value-added / non-value-added activities

Case Study

Errors in batch records were reduced

29.7%

Errors in batch records were reduced

29.7%


29.7%29.7%

Efficiency Metrics – For the first time in QA


Case Study-2.1: Efficiency Gain: Earned

Hrs. Since Smart-QC Kick off

223.2

258.1267

279.2 282.6

240

260

280

300

Eff

icie

ncy [

%]

Earned Hrs. Per WeekSince Smart-QC Go-Live

Earned Hrs. increases by


192.5

212.5

223.2

180

200

220Eff

icie

ncy [

%]

Earned Hrs

Earned Hrs = Sum of ALL Test related activities times the STD

Earned Hrs. increases by

almost 50%

42.4%

36.4%41.3%

59.7%63.6%

55.4%

40.8%

72.6%

93.8%

83.1%

70.1%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

300

400

500

600

700

800

900

1,000

Ax

is T

itle

Efficinecy & Earned Hrs for past 12 wks




27.1%

0.0%

10.0%

20.0%

30.0%

0

100

200

300

Axis Title

Earned Hrs Efficiency

30

Earned Hrs = Sum of ALL Test related activities times the STD

The average efficiency for the last 4 weeks is 79.9% while the first 4 weeks of

that 12 weeks period was 36.8%



Week Date Avail Hrs Unavail HrsNon Test

HrsRemaining Hrs Earned Hrs Unaccounted Hrs

% Earn Hrs Of

OverallEfficiency

35 29-Aug-10 1,718 323 461 934 655 279 38.1% 70.1%

34 22-Aug-10 1,712 270 582 861 715 145 41.8% 83.1%

33 15-Aug-10 1,552 220 397 934 876 58 56.4% 93.8%

32 8-Aug-10 1,535 217 411 906 658 248 42.9% 72.6%

31 1-Aug-10 1,522 237 402 883 360 523 23.7% 40.8%

30 25-Jul-10 1,521 174 433 914 507 407 33.3% 55.4%


30 25-Jul-10 1,521 174 433 914 507 407 33.3% 55.4%

29 18-Jul-10 1,557 120 612 825 525 301 33.7% 63.6%

28 11-Jul-10 1,529 189 665 675 403 272 26.3% 59.7%

27 4-Jul-10 1,535 321 479 736 303 432 19.8% 41.3%

26 27-Jun-10 1,914 250 586 1,077 292 786 15.2% 27.1%

25 20-Jun-10 1,906 315 617 974 354 620 18.6% 36.4%

24 13-Jun-10 1,902 319 603 980 415 565 21.8% 42.4%

Standard Smart-QC Report: The average efficiency for the last 4 weeks is 79.9% while the first 4

weeks of that 12 weeks period was 36.8%

THANK YOUTHANK YOU


Web Site: www.cManageEfficiency.comEmail: [email protected] Web Site: www.cresultsconsulting.comContact Info: Rafi Maslaton; 1-732-748 1990 ext. 250

kpi & dashboard in quality operation - smart-qc · pdf filecontinuous improvement...

Documents