medical device quality metrics
TRANSCRIPT
![Page 1: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/1.jpg)
Medical Device Quality Metrics
MDIC Case for Quality Forum October 26, 2016
FDA/Xavier University Metrics Initiative
![Page 2: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/2.jpg)
Identify Meaningful Metrics for Industry
To provide a system of metrics across the Total Product Lifecycle that:
1. Informs internal company decisions and triggers action
2. Shifts the Right-First-Time mentality closer to the initial days of development
Purpose and Outcome
![Page 3: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/3.jpg)
Assessment Across the TPLC
3
Pre-Production Production Transfer
Production Continual Improvement
& Risk Mgmt.
Enterprise-Wide Continual Improvement
R&D Continual Improvement
& Risk Mgmt.
Post-Production
![Page 4: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/4.jpg)
Top 3 Metrics Finalized
Phase/Metric Name/Goal Metric Calculation
Pre-Production: Design Robustness Indicator
Assess the number of product changes that are related to product or process inadequacies or failures
Total # of product/process changes across projects
Total # of projects
Production: Right First Time Rate
Assess the number of production failures related to product and process inadequacies or failures
# of units mfg. Right-First-Time within or across lots
# of units started
Post-Production: Post-Market Surveillance
Assess an aggregate of post-market indicators with root causes of product or process inadequacies or failures
1. Basic Level: We provide proven calculations for many post-market indicators.
2. Advanced Level: We provide an example of how
to factor risk scores into the post-market indicators.
4
![Page 5: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/5.jpg)
“Best Practices” Document
• Metric output can be used to understand root causes
• Combine metric output with other metrics to understand a more holistic picture and analyze trends
• Goal is to provide a feedback loop to improve systems from the earliest point possible that allowed the failure to occur originally
5
Purpose: To help organizations understand how best to use the output from the metrics to inform decisions and trigger actions
http://xavierhealth.org/device-measures-initiative/
![Page 6: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/6.jpg)
Enterprise-Wide Assessment
6
Pre-Production Production Transfer
Production Continual Improvement
& Risk Mgmt.
Enterprise-Wide Continual Improvement
R&D Continual Improvement
& Risk Mgmt.
Post-Production
![Page 7: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/7.jpg)
Imagine: TPLC Risk Analysis
Y-axis = Internal Risk Score
Each Point = Risk Across TPLC
“Internal” includes pre-production and production metric total risk score
X-axis = External Risk Score
“External” includes the total post-production risk score of appropriate indicators
![Page 8: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/8.jpg)
Next Steps - Industry
No need to identify and pilot additional metrics
– Our Best Practices Document provides guidance on how to use metrics to inform decisions
– We believe the 3 metrics chosen provide great insight on product risk when used as we describe (including the Enterprise-Wide roll-up), but companies will have many more they are monitoring or alternate versions of these 3 that make sense for their business operations
8
![Page 9: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/9.jpg)
Next Steps - Industry
Focus on quantitating the strength of an organization’s Culture of Quality
• Why? Recognition that lowering risk to product quality can only be sustained in a strong Culture of Quality
• How? Use the Xavier/PwC Culture of Quality Alignment Optimization Assessment – What would you have to see in action to know a company has a strong
culture of quality?
– Schellingpoint process uses complex algorithms determine alignment and sentiment through a 2-step process that takes a total of 15 minutes
– Results are verifiable
– Provides the state of maturity of the organization
9
![Page 10: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/10.jpg)
Next Steps - FDA
We continue to believe these and all performance metrics need to be assessed in context
– This can be accomplished on inspection
– Otherwise could lead to false conclusions and unintended consequences
10
![Page 11: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/11.jpg)
ORA Perspective on Metrics Use
11
Collaborative Effort
(multiple relevant sources of info)
*Presented by Ellen Morrison at FDA/Xavier PharmaLink Conference, March 2016.
![Page 12: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/12.jpg)
ORA Perspective on Metrics Use
12
*Presented by Ellen Morrison at FDA/Xavier PharmaLink Conference, March 2016.
![Page 13: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/13.jpg)
2014 – 2016 Deliverables
1. List of 97 Gold and Silver activities that are above compliance across 11 critical systems and 3 phases of production
2. Identification of 17 measures linked to impact to patient safety, design robustness, process reliability, quality system robustness, and failure costs
3. Conversion of 3 measures into defined metrics
4. 2 year Retrospective Pilot Study completion and analysis
5. “Best Practices” Metric Output Documents
13
![Page 14: Medical Device Quality Metrics](https://reader031.vdocuments.us/reader031/viewer/2022021405/6209377f9c2bc374b531c78a/html5/thumbnails/14.jpg)
Kristin McNamara
Senior Advisor to DACRA Office of Regulatory Affairs FDA
Marla Phillips
Director Xavier Health Xavier University