risk assessment tools for ivd assays
TRANSCRIPT
Risk Assessment Tools for Identifying Hazards and
Evaluating Risks Associated with
IVD AssaysRobert C. Menson, PhD
AACC Expert AccessLive on Line06 July 2004
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 2
What Risks Must Be Managed?
Business
Product liability
Regulatory
Risk to safety of patients, users, handlers
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 3
Elements of the Risk Management Process
Risk Analysis
Risk Evaluation
Risk Control
Post-Production Information
Ass
essm
ent
Man
agem
ent
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 4
Risk Assessment Tools
Risk MatrixPHA= Preliminary Hazard AnalysisFTA=Fault Tree AnalysisFME(C)A=Failure Mode Effects (Criticality) AnalysisHAZOP=Hazard Operability AnalysisHACCP=Hazard Analysis and Critical Control Point
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 5
Applications of Risk Analysis
DesignProductionPre-market NotificationsComplaintsMDRChange ControlFailure AnalysisEtc.
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 6
Risk ManagementLife Cycle
Concept & Feasibility Development Scale-Up & Transfer Production
System Level Assessment
Design Assessment
Process Assessment
Customer FeedbackAssessment
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 7
Risk Management& Design Control
Scale-Up & Transfer
DesignControl
RiskAssessment
Requirements Plan SpecificationsTest
Methods &Results
ProductionsMethods
Change Records
RiskManagement
Plan
PreliminaryHazard
Analysis
Detailed Analysis(FMEA, FTA, HACCP, HAZOP)
RiskManagement
Report
Risk Reviews
ProductionDevelopmentConcept& Feasibility Planning
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 8
Key Concepts of Risk
The frequency of the potential harm;How often the loss may occur;
The consequences of that loss;How large the loss might be;
The perception of the loss;How seriously the stakeholders view the risk that might affect them.
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 9
Risk Management Terms
Intended Use/PurposeHarmHazardRiskResidual RiskTolerable Risk
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 10
Intended Use/Purpose
Use of a Product, Process or Service in accordance with the specifications, instructions and information provided by the manufacturer
ANSI/AAMI/ISO 14971:2000, definition 2.5
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 11
Harm
Physical injury or damage to health of people, or damage to property or the environment.
ISO/IEC Guide 51:1999, definition 3.3“Guidelines for inclusion of safety aspects in
standards.”
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 12
Hazard
Potential source of Harm
ISO/IEC Guide 51:1999, definition 3.5
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 13
Risk
Combination of the probability of occurrence of harm and the severity of harm.
ISO/IEC Guide 51:1999, definition 3.2
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 14
Residual Risk
Risk remaining after protective measures have been taken.
ISO/IEC Guide 51:1999, definition 3.9
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 15
Tolerable Risk
Risk which is accepted in a given context based on the current values of society.
ISO/IEC Guide 51:1999, definition 3.7
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 16
Hazard Identification
HazardIdentifcation
BrainstormingPHAFTA
FMEA
LawsCodes
Standards
EventsMDRs
AccidentsEtc
Listof
Hazards
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 17
Risk Matrix
HIITOccasional
IILTRemote
HHILProbable
HHILFrequent
SevereMajorMinorNegligible
SeverityFrequency
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 18
Severity of Consequence Codes
Severity Nuclear Medical Generic
Severe Virtually complete loss of plant.
-Many fatalities
Potential death or Permanent injury
Product must be scrapped.Loss of customer or repeat business.
Compliance in jeopardy.
Major Extensive damage to plant.
Few fatalities
Potential serious injury.
Not permanent
Major service or downtime.Significant rework.
Customer will notice.May cause injury.
Loss of customer confidence
Minor Significant damage to plant. Severe injury or
occupational illness
Potential minor injury.
Not permanent
Reduced service.Minor rework or re-inspection.
Customer may notice.
Negligible Minor systems damage.
Minor injury or occupational
illness
Potential minor discomfort.
Not permanent.
Minor nuisance.No effect on service.
Customer won’t notice.
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 19
Frequency Codes
Frequency Nuclear Medical Generic
Frequent > Once/yr >50% of time Continual occurrences
Probable 0.1-1/yr 10-50% of time Occurrences are frequentHappens, probable reoccurrence
Occasional 0.01-0.1/yr 1-9.9% of time Isolate occurrences happen
Remote 0.0001-0.01/yr 0.1-0.99% of time Isolated occurrences possibleDon’t expect reoccurrence
Improbable 0.000001-0.0001/yr 0.0001-0.1% of time
Not likely
Incredible <0.000001/yr<1 in 1,000,000
yrs
<.0001% of time<1 in 10,000
Incredible
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 20
Estimation of Risk Codes
H: High I: Intermediate
L: Low
T: Trivial
Risk must be reducedReduced to ALARP-cost a minor factor
Reduce to ALARP-consider cost/benefitBroadly acceptable
ALARP=As Low As Reasonably Possible
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 21
PHA
IOMaQSR TrainingSOPsInternal Software Checks
Installation of Incorrect Software Version
TRMiBattery life certificationDrain testingPreventive maintenance
Loss of bits on SRAM due to weak battery
IOMiSoftware Qualification TestingBackup Sample
Software bug/failure
LRMiReadily apparent to operatorSample can be reprocessedBackup sample
Pump, Valve or Liquid Sensor Malfunction
TRNStress/Simulated Life Cycle Testing8-Hour “burn in”
System failure due to deterioration over life cycle
Impact
Freq.Sev.Investigation/ControlsHazardHazards Arising From Functional Equipment Failure
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 22
ALARP Curve
Increasing Severity of Harm
Incr
easi
ng P
roba
bilit
y of
Occ
urre
nce
Intolerable Region
ALARP
BroadlyAcceptable
Region
MaximumTolerable
Risk
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 23
FMEA vs FTA
Assumes failure of the functionality of a productIdentifies part/module failure as cause of functional failure
Assumes component or part failureIdentifies functional failure as a result of part failure
FTAFMEA
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 24
FMEA vs. FTADone for entire design
Systematic way to predict new problems
A bottoms-up analysis
People expect the same results from FTA which is not true
Too difficult to do for entire designSystematic way to predict causes for problems
A top down analysis
People do not expect the same results from and FMEAOften a fault tree is used for a problem or an accident.
FMEA FTA
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 25
FTAAssumes fault and analyzes possible causesConnection tool for PHA* to subsystems or modulesTop downDeductiveEvaluate system (or subsystem) failuresConsidered more structured than FMEAGraphical presentation--visual picture*Preliminary Hazard Analysis
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 26
FailuresPrimary--Due to internal causes that include poor design or use of inappropriate materialsSecondary--Due to failures in the operation that include equipment failureControl--Due to failures in the systems that are in place to protect the quality and safety
e.g. raw material outside specificationfailure of safety switchfailure of test method
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 27
FTA Basic SymbolsBasic Flow
FAULT
AND
OR
Fault in a box indicates that it is a result of subsequent faults
Connects a preceding fault with a subsequent fault that could cause a failure
Connects two or more faults that must occur simultaneously to cause the preceding fault
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 28
FTA During Design P O O R F ITIN H A N D
D O E S N O TR E S T R IC TS P IL L A G E
O R
O R
A S P E C T R A T IOT O O H IG H
M A J O RD IA M E T E R
T O O L A R G E
A S P E C T R A T IOT O S M A L L
B A S EF L A T N E S S
B A S E S IZ ET O O S M A L L
R IM T O F IL LIN S U F F C IE N T
CompetingDesign Criteria
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 29
FTA Lab Failure
Patient Death OR
Other
Lab Error OR
Outliers
Systematic
Random
OR
CalibrationError
Interference
Other
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 30
FTA During ReliabilityHAZARD
SYSTEMFAILURE
DRIFT>LIMIT
CMPT A FAILS CMPT B FAILS CMPT C DRIFTS REFERENCEDRIFTS
AN
D
OR
OR 4. x 10-9
4. x 10-91. x 10-16
3. x 10-9 1. x 10 -92. X 10 -85. x 10 -9
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 31
What is FMEA?What is FMEA?
FMEAFailure Mode and Effects Analysis
FMECAFailure Mode Effects and Criticality Analysis
FMEAFailure Mode and Effects Analysis
FMECAFailure Mode Effects and Criticality Analysis
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 32
What is FMEA?
Powerful prioritization toolInductiveHigh effective tool for identifying critical quality attributesHigh structuredMethodicalBreaks large complex designs into manageable steps
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 33
FMEA
Bottom up approachEvaluates specific failuresDetailed analysis tool
Use in conjunction with PHA and FTA Complements FTA
May lead to different failure results
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 34
Advantages of FMEA
Less analyst dependent than FTAAllows direct criticality assessment of componentsValuable troubleshooting aidIdentifies areas of weak designIdentifies areas of high riskPrevention planningIdentifies change requirements
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 35
Disadvantages of FMEADoes not consider operator errorTediousMay not apply to all systems--especially softwareMay require extensive testing to gain informationMay miss some failure modes
Time pressuresInformation missing
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 36
DefinitionsCriticality -- Weighting of hazard severity with the probability of failureSeverity -- Seriousness of effect through its impact of the system functionOccurrence -- Likelihood a specific failure will be caused by a specific cause under current controls, i.e. the probabilityVerification -- Ability of the current evaluation technique to detect potential failure during designDetection -- Ability of the current manufacturing controls to detect potential failure before shipping
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 37
DefinitionsDefinitions
Risk Priority Number (RPN)
= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)
Risk Priority Number (RPN)
= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 38
FMEA ExampleP O T E N T IA L F A IL U R E M O D E A N D E F F E C T S A N A L Y S IS
Im u n n o a s s a y D e v ic e F M E A N u m b e r S u b s y s te m P a g e o f X C o m p o n e n t _ S u p p o rt_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ R e s p o n s ib ility P re p a re d B y P ro c e s s F M E A D a te (O rig .) (re v .) C o re T e a m : _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
D e vic e / P o te n tia l P o te n tia l S P o te n tia l O C u rre n t D R R e c o m m e n d e d R e s p o n s ib ilit
y A c tio n R e s u lts
F u n c tio n F a ilu re E ffe c t(s ) C a u s e (s ) C o n tro ls P A c tio n (s ) a n d T a rg e t A c tio n s R M o d e o f F a ilu re o f F a ilu re N C o m p le te D a te T a k e n S O D P
N S u p p o rt S y s te m
H o ld s A n tib o d y
-
S e p a ra te s A n a ly te
F a ils to S e p a ra te a n a ly te
N o A n s w e r E rro n e o u s R e s u lt F a ls e H i F a ls e L o
4
79
W ro n g M a te ria l
3
33
5
55
6 0
1 0 5 1 3 5
D e v e lo p s C o lo r
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 39
Process FMEA Process FMEA Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for
reducing occurrence or increasing detection of the failure conditions
Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for
reducing occurrence or increasing detection of the failure conditions
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 40
Sources of Process Defects
Sources of Process Defects
Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts
Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts
Adjustment errorAdjustment errorProcessing wrong Processing wrong work piecework pieceMisMis--operationoperationEquipment not set Equipment not set up properlyup properlyTools and fixtures Tools and fixtures improperly preparedimproperly prepared
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 41
FMEA Summary
Powerful tool for summarizing:Important modes of failureFactors causing these failuresEffects of these failuresRisk prioritizationIdentifying plan to control and monitorCataloging risk reduction activities
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 42
HAZOP
Haz ard and Op erability StudyBottom up analysisDeviations from design intentionsSystematic brainstorming based on guide words
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 43
HAZOPGuide Words
No/NotMoreLessAs well asOther than
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 44
Hazop ModelDesign Statement
Activity Material Destination
Transfer Powder Hopper
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 45
HAZOP
LiquidWrong powder
Other than
Larger tankInaccurate gage
Pump fastMore
Valve closedHopper full
Tank emptyValve closedLine blockedPump broken
NoHopperPowderTransfer
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 46
HAZOP Plan
Interlock
Operator trainingPM
Low
Med
Med
Valve closedLine blockedPump broken
Powder flow
NO
WhoActionRiskCausesDeviationGuide
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 47
HACCP
Risk Management SystemBiological HazardsChemical HazardsPhysical Hazards
RequiresPrerequisite Quality System Program Traditionally GMPs
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 48
HACCP Steps1. Conduct hazard analysis and identify
preventive measures2. Identify Critical Control Points.3. Establish critical limits4. Monitor each critical control point5. Establish corrective action to be taken when
deviation occurs6. Establish verification procedures7. Establish record-keeping system
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 49
HACCP Decision TreeP re v e n tiv eM e a s u re s
E x is t
S te p E lim in a teo r R e d u c e
C ritic a lC o n tro l P o in t
Is c o n tro lN e c e ss a ry fo r
S a fe ty
S to pN o t a C C P
M o d ify S te p
N o
N o
Y e sY e s
Y e s
E x ce s sC o n ta m in a tio n
P o s s ib le
N o
N o
S u b s eq u en tS te p s e lim in a te
o r R e d u c eY e s
Y e s
N o
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 50
HACCP Worksheet
Biological
Chemical
Physical
Is this step a critical control point? (Y/N)
What preventative measures can be applied to prevent
the significant hazards?
Justify your
decisions for column
3.
Are any potential safety
hazards significant?
(Y/N)
Identify potential hazards
introduced, controlled or
enhanced at this step(1)
Material/processin
g step
654321
Product Description:
Method of Storage and Distribution
Intended Use and Consumer
Firm Name:
Firm Address:
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 51
HACCP Plan
(7) Who
(6)Frequency
(5)How
(4)What
(10)Verificat-
ion
(9)Record
s
(8)Correct-
iveActions
Monitoring
(3)Critical
Limits for each
Action
(2)Significant Hazards
(1)Critical Control Point
Product Description:
Method of Storage and Distribution:
Intended use and Consumer:
Firm Name:
Firm Address
© 2004 Menson & Associates, IncAACC Expert Access
Live on Line 52
Contact Information
Robert C. Menson, PhD
Menson & Associates, IncQuality Systems and Compliance
For FDA Regulated IndustriesPO Box 480
Newport, RI 02840Phone: 1.401.847.6320
Email: [email protected]