quality systems: managing human error and capa effectiveness john shaeffer may 2008

Quality Systems: Quality Systems: Managing Human Error Managing Human Error and CAPA Effectivenessand CAPA Effectiveness

John Shaeffer

May 2008

Objectives

Why is "Not following SOPs" a top issue for the pharmaceutical industry

How to know if your CAPA will be effective

How to manage human error within

systems

Top 10 Drug Observations (FDA) 2008

Total # of Citations Citation Text1. 734 – Responsibilities and procedures in QC not in writing/ followed2. 629 – Written production & process control procedures not followed3. 532 – Control procedures not established which monitor/validate

manufacturing4. 457 – Lab controls do not include appropriate test procedures5. 451 – No written procedures for production and process controls6. 423 – Testing and release do not include appropriate lab results7. 422 – Failure to fully review discrepancies to determine whether or

not the batch has been thoroughly distributed8. 401 – Employees are not given training in operations/procedures

being conducted9. 400 – Batch production and records do not include complete

information10. 344 – Written procedures not established/followed for the cleaning

and maintenance of equipment (including utensils)

As of 02/11/2008

Revision Process: Routine Violations

Undefined procedure structure/usage rules

+

No “on the spot correction” process

+

Slow normal revision cycle

times+

High frequency of internal

revisions=

ROUTINE VIOLATIONS

Execution Excellence Goals

Prevent EVENTSIncrease SOP adherence

Increase CAPA effectiveness

Increase efficienciesImprove product cycle times

Decrease inadvertent errors

Addressing Events - CAPAs

Typical Organizational Progression

Blind Eye Awareness Accountability

Shared Responsibility

Consequences!

Events

vs.

Errors

Latent

vs.

Active

Error Types

Typical CAPAs for Personnel Events

Retrain

Counsel

Isolated incident - no action required

Increased supervision

Disciplinary action up to dismissal

Typical CAPAs for Procedural Events

Fix the procedure Add a note or caution

Add another step to specifically address the event

Fix the person Counsel

Retrain

No action

Increased supervision

Disciplinary action

Underlying PrinciplesPeople:

Are FALLIBLE Achieve high levels of performance based largely on the

ENCOURAGEMENT and REINFORCEMENT received from leaders, peers and subordinates

Will exhibit behaviors that are influenced by ORGANIZATION PROCESSES and VALUES

Error likely situations are: Predictable Manageable Preventable

Events can be prevented: Understanding the reasons mistakes occur Applying the lessons from past events

Human Performance = Results + Behaviors

AutomobileManufacturer

OrganizationalIssues

Flawed Defenses

Initiating Action

Error Precursors

James Reason Ph. D., 1990

ORGANIZATIONAL CULTURE

RESPONSE to inefficiency and error is related to organizational culture.

An organization's culture is reflected by what it does: Practices

Procedures

Processes

An organization's culture is NOT what it claims to espouse or believe in.

ORGANIZATIONAL CULTURE3 Types:

1. PATHOLOGIC; the organization says “We don't make errors, and we don't tolerate people who do.” likely to shoot the messenger

2. BUREAUCRATIC:

write a new rule

3. LEARNING ORGANIZATION

seeks to understand the broader implications of error

Ron Westrum Ph. D., 1984

Time

Typical Procedure Progression - Bureaucratic

Actions allowed by plant procedures

Actions requiredto perform the job

Non Compliance

Compliant Operations

Events

LEARNING ORGANIZATION

Actively seek flaws in systems Messengers are trained and rewarded Responsibility is shared Failures lead to far-reaching reforms New ideas are welcomed

Creating an organizational culture that supports OPEN

DISCUSSION of errors & near misses is perhaps the

SINGLE MOST EFFECTIVE INTERVENTION.

As such a culture is created,THE ERROR RATE WILL

INCREASE, not because more are made, but because

MORE ARE REPORTED.

Errors & Organizational Culture




Flawed Defenses

Initiating Action

Error Precursors


Defenses

Physical barriers to control the processExamples:

- Policies, Procedures and Job Aids

- Alarms, Warning signs, Labels and Floor Markings

Flaws in defenses Promotes errors

Creates error likely situations

Procedure Reading Test

PARISIN THE

THE SPRING

BIRDIN THE

THE HAND

ONCEIN A

A LIFETIME

Procedure Reading Test

What did you see?

PARISIN THE

THE SPRING

BIRDIN THE

THE HAND

ONCEIN A

A LIFETIME

Humans will ….

not see what is there and

see what is not there

i cdnuolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg. The phaonmneal pweor of the hmuan mnid; aoccdrnig to a rscheearch at camabrigde Uinervtisy, it dseno't mtaetr in waht oerdr the ltteres in a wrod are, the olny iproamtnt tihng is taht the frsit and lsat ltteer be in the rghit pclae.

The rset can be a taotl mses and you can sitll raed it whotuit a pboerlm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. Azanmig huh? yaeh and I awlyas tghuhot slpeling was ipmorantt!

1Can you raed tihs?

1Cambridge University




Flawed Defenses

Initiating Action

Error Precursors


Task

Demands

Error PrecursorsTWIN Analysis

Work

Environment

Individual

Capabilities

Nature

(human nature)

complicated vs. simple

time constraints

multiple steps at the same time

lighting

noise

clothing requirements

space

first time evolution

training

how long since last performed

personal affects

shortcuts

egos

culture

time into shift

perceived pressures

Human Nature Effects

Take 1000 and Add 40 to it. Now add another 1000. Now add 30. Another 1000. Now add 20. Now add another 1000. Now add 10. What is the total?

Add these numbers (without a calculator!)

Answer

Did you get 5000? Great job! You probably followed the procedure usage rules

discussed earlier.

You are not alone! Did you get 4010 or 410? If you said 4100 great job!

However, chances are you violated the procedure usage rules discussed earlier.

The decimal sequence confuses our brain, which always jumps to the highest decimals (100s instead of 10s).

Human Nature Effects

Take 1000 and Add 40 to it. Now add another 1000. Now add 30. Another 1000. Now add 20. Now add another 1000. Now add 10. What is the total?




Flawed Defenses

Initiating Action

Error Precursors


Operational Modes (Initiating Action)

Attention

Low

High

Low High

Skill BasedER = 1 in 10,000

Rule BasedER = 1 in 1,000

Knowledge Based

NOTE: Error Rate (ER)

ER = 1 in 2

TaskDemands

Importance of Operation Modes

Determines corrective action

Dictates procedure usage rules

Identifies error likely situations

Determines efficiency of execution


Attention

Low

High

Low High



Knowledge Based


ER = 1 in 2

TaskDemands




Flawed Defenses

Initiating Action

Error Precursors


Performance Problem CAUSES26 Leading Organizations Surveyed

Airlines Delta Airlines Jet Blue

Automotive Lexus

Financial Services & Banking American Express Barclays Bank

Government Navy New York Fire Department Social Security Administration

Manufacturing Caterpillar Steelcase Molex Johnson Controls

Media BBC

Retail Coffee Ben & Tea Leaf Godiva SAB Miller

Pharmaceutical and Healthcare Becton Dickson Eli Lilly Humana Novartis

Industrial Agilent

Telecom and Technology Nextell IBM SAS SAP

Results: 24.3% Individual 75.7% Environmental

Tony O’Driscol Ph. D., 2006

Reliance on Employee vs. CA Effectiveness

HIGH

Low

HighLOWReliance on Employee

Effectiveness

Desired area for CAPA

CAPA Effectiveness: Targeting Human Performance Problems

100 events

75% of events are outside of individuals control

5 events

Assume corrective actions are 20% effective

Number of eventswith effective corrective actions

25 events

100 Total Events

Procedure Checklist: Procedure GMP-10052, Documentation Of GMP Practices

Step Performed By:

Checked By:

1 One action per step JS AP

2 Every step to successfully complete the task should be in the procedure JS AP

3 Procedure steps should be in the order that they are performed JS AP

4 Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS

5 Only information critical for step implementation should be included within the step JS AP

6 Use note for location if a particular piece of equipment is hard to find AP

7 Use cautions to delineate steps that need to be done together in a rapid fashion or if time is critical

JS AP

8 Use bullets or numbering in accordance with site standards JS AP

9 Use specific nomenclature for identifying components to be manipulated when possible JS AP

10 Place visuals close to the relevant text

11 The range of acceptable values for an entry should be next to or below the entry JS AP

12 Critical steps should be identified JS AP

13 One series of signoffs per step (performed by, checked by, verified by) JS

14 Batch record signatures, checked by and verified by should be aligned in separate columns AP

15 Any cautions or important notes should be in the procedure before the information is required to be used

JS AP

16 Write the WARNING: statement to the person using the SOP JS AP

17 Align the second line of the warning statement with the text and not the word JS AP

Performed By: _____John Shaeffer__________________________ Date:________09/24/06____ Checked By: _____Amy Peterson____________________________ Date:________ 09/24/06____


Step Performed By:

Checked By:







7 Use cautions to delineate steps that need to be done together in a rapid fashion or if time is critical JS AP








15 Any cautions or important notes should be in the procedure before the information is required to be used JS AP



Performed By: _____John Shaeffer__________________________ Date:________09/24/06____

Checked By: _____Amy Peterson____________________________ Date:________ 09/24/06____

How many missing initials?


Step Performed By:

Checked By:







7 Use cautions to delineate steps that need to be done together in a rapid fashion or if time is critical JS AP








15 Any cautions or important notes should be in the procedure before the information is required to be used JS AP



Performed By: _____John Shaeffer__________________________ Date:________09/24/06____

Checked By: _____Amy Peterson____________________________ Date:________ 09/24/06____

Ready to do it again?


One action per step JS AP Every step to successfully complete the task should be in the procedure JS AP Procedure steps should be in the order that they are performed JS AP Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS ___ Only information critical for step implementation should be included within the step JS __ Use note for location if a particular piece of equipment is hard to find.___ AP Use cautions to delineate steps that need to be done together in a rapid fashion, if time is critical, or an error likely situation JS AP Use bullets or numbering in accordance with site standards JS AP Use specific nomenclature for identifying components to be manipulated when possible JS AP Place visuals close to the relevant text The range of acceptable values for an entry should be next to or below the entry JS AP Critical steps should be identified JS AP One series of signoffs per step (performed by, checked by, verified by) JS __ Batch record signatures, checked by and verified by should be aligned in separate columns ___ AP Any cautions or important notes should be in the procedure before the information is required to be used JS __ Write the WARNING: statement to the person using the SOP JS AP Align the second line of the warning statement with the text and not the word JS AP Performed By: John Shaeffer Date: 09/24/06 Checked By: Amy Peterson Date: 09/24/06

How many missing initials?


One action per step JS AP Every step to successfully complete the task should be in the procedure JS AP Procedure steps should be in the order that they are performed JS AP Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS ___ Only information critical for step implementation should be included within the step JS __ Use note for location if a particular piece of equipment is hard to find.___ AP Use cautions to delineate steps that need to be done together in a rapid fashion, if time is critical, or an error likely situation JS AP Use bullets or numbering in accordance with site standards JS AP Use specific nomenclature for identifying components to be manipulated when possible JS AP Place visuals close to the relevant text The range of acceptable values for an entry should be next to or below the entry JS AP Critical steps should be identified JS AP One series of signoffs per step (performed by, checked by, verified by) JS __ Batch record signatures, checked by and verified by should be aligned in separate columns ___ AP Any cautions or important notes should be in the procedure before the information is required to be used JS __ Write the WARNING: statement to the person using the SOP JS AP Align the second line of the warning statement with the text and not the word JS AP Performed By: John Shaeffer Date: 09/24/06 Checked By: Amy Peterson Date: 09/24/06


Attention

Low

High

Low High



Knowledge Based


ER = 1 in 2

TaskDemands

Predicted Results

First documentSpeed – 7 seconds average

Success rate – over 85%

Second documentSpeed – doubled in time (15 seconds)

Success rate – reduced significantly (under 10%)


Flawed Defenses

Operation Modes- Skill- Rule- Knowledge

Error Precursors

James Reason Ph. D., 1990, modified

Performance Model

Procedure PolicyJob Aid LabelAlarm Sign

Task Demands

Work Environment

Individual Capabilities

Nature (human nature)

Initiating Action

Process & valuesSystem alignmentCommunicationBehavior (Culture)

20% Individual

80% Organizational

CA 5% CA 95%

CA – Corrective Action

“Defense in Depth Model” : Anatomy of Events

HumanFallibility

Flawed Defenses

OrganizationalBarriers

ManagementBarriers

InitiatingAction

Modified from Managing the Risks of Organizational Accidents, James Reason, Pd. D. 1997.

ACTIVE ERRORS

- Weak Skills- Failed or Non- existent Barriers

LATENT ERRORS

Organizational issues:- Poorly Written Procedures- Failed or Nonexistent Programmatic Barriers - Ineffective Management

Event

Corrective Actions Effectiveness

ELIMINATE: Organizational Issues

ELIMINATE: Flawed Defenses & Error

Precursors

PROVIDE: Preventative Actions for the

individual who initiates the

action

More

Less

Execution Excellence

Nine different components1. HPM Basic Concepts

2. HPM Based Investigations

3. Human Factoring Records

4. Human Factoring Processes

5. Error Prevention Techniques

6. Preventive Actions

7. Behavior Modification

8. Organizational Issues

9. Procedure Usage

QUESTIONS?

quality systems: managing human error and capa effectiveness john shaeffer may 2008

Documents