how to apply deming’s philosophies in the world of …...sd, cv% 14 adverse product infusion...
TRANSCRIPT
![Page 1: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/1.jpg)
1
Shadya George, MLS(ASCP)CM
GMP Quality Manager Cell Therapy Manufacturing
![Page 2: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/2.jpg)
2
Why are We Here?
*
![Page 3: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/3.jpg)
3
Purpose & Use of Quality Indicators
Critical Areas to Monitor
Indicator Types & Desired Characteristics
Design of Balanced Monitoring Program
Metrology (Measurement) Selection
SPC Analysis – Statistical Process Control
Ground Up Challenges & Strategies
![Page 4: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/4.jpg)
4
Provide Evidence-Based Quality Assurance, Risk Prediction, & Corrective Action
![Page 5: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/5.jpg)
5
Predict Acceptability of Clinical Outcomes
Treatment/Collection Decisions
Predict Risk of Adverse Reactions
Development of Process Understanding
Process Improvement/Optimization (DOE)
Alerts for Needed Corrective Action
Regulatory Compliance
![Page 6: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/6.jpg)
6
Key Process Areas for Optimization
• Potency
• Purity
• Identity
• Efficacy
• Cell Dose Enumeration
Technical Manipulations: Cell Loss/Recovery
Clinical Outcomes: Engraftment
Regulatory Based Priorities:
• Historic Weak Compliance Areas
Critical Areas of Risk: FMEA
![Page 7: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/7.jpg)
7
Team Brainstorming Process Failure Risk Study:
List Possible Process Failures Assign Risk Score
Severity
Likely/Frequency of Occurrence
Detectable
![Page 8: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/8.jpg)
8
Balance a Mix of Types of Indicators based on
your Process and System Risks
![Page 9: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/9.jpg)
9
![Page 10: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/10.jpg)
10
# Name Categor
y Level Type Dimension
Department Responsibility
Definition Formula Units Data
Location Collection Frequency
Report Type
16 DMSO
Exposure Time Internal Process
Activity Process Safety Manufacturing Time from addition of DMSO
freeze media to cells until step advancement for freeze
protocol
Time of freeze initiation - Time of DMSO addition min
Product Batch
Record
Quarterly Run Chart
17 Cell Viability Post Thaw
Internal Process
Activity Process Effective Manufacturing % of living cells as determined
by validated & controlled release testing assay
(# viable cells/total cells)100%
Average product type viability, SD, CV% Annually
% Product Batch
Record
Quarterly
Run Chart
Bar Graph with Annual Averages, SD, CV%
14
Adverse Product Infusion
Reactions
Internal Process
Activity Outco
me Safety Clinical
Graded seriousness of reactions from 0 for no
reactions to 5 for complications resulting in patient death.
Charted for each infusion on graph quarterly, calculate
average, standard deviation and %CV annually
Average: Sum of all infusion reaction grades/total # infusions performed within calendar year SD: %CV: (SD/Average)100%
Grade/Infusion
Annual Grade Average
Annual Grade
SD
Annual Grade %CV
Product Infusion
Sheet Quarterly
Run Chart,
Scatter Plot linear Correlation against unit characteristics
Annually
Patient Mortality
Internal Process
System Outco
me Safety Clinical
Tracking % patient
deaths for any cause
within 1 year of treatment
(# Patient Deaths/Total
Patients Treated)100%
% Clinical
Outcome Report
Annual Bar
Graph
Manufacturing Cost
Finance Activit
y Struct
ure Efficient Manufacturing
Average Cost to manufacture 1 unit, to include supplies, labor,
release testing
Sum of (manufacturing supplies used, testing services, testing
supplies, office supplies, proportion of equipment & facility certification costs, &
labor)/#Units manufactured. Performed for each individual
product type.
$/unit Finance
Department
Annual Run
Chart
![Page 11: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/11.jpg)
11
Can We Monitor Too Much?
![Page 12: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/12.jpg)
*What type of processes do you
perform?
Autologous HPC
Allogeneic HPC
Clinical Trial Cell Manufacturing
All of the Above
12
![Page 13: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/13.jpg)
*How many indicators do you track
on average for each manufacturing
processes?
1-4
5-10
11-20
>20
13
![Page 14: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/14.jpg)
*How many indicators total are
tracked and reported quarterly
within your facility?
1-9
10-24
25-50
>50
14
![Page 15: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/15.jpg)
15
Benefit vs Time & Cost Considerations
Risk Loss Focus on Critical Areas - CLUTTER
Frequency Decisions? How Often?
Should Our Indicators of Focus Change?
Effective Quality Improvement
Requires Prioritizing!!
![Page 16: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/16.jpg)
*How frequently do you revise your
monitoring system (adding/omitting
indicators)? Quarterly
Annually
Rarely (> 5 years)
Never done it 16
![Page 17: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/17.jpg)
17
Feasible Valid Understandable Specific Sensitive Reliable: Repeatability, Robustness, Reproducibility Mainz, J. (2003). Defining and Classifying Clinical Indicators for Quality Improvement. International Journal for Quality in Health Care, 15(6), 523-530.
![Page 18: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/18.jpg)
18
How can my indicator be
valid if I am not sure my
measurement is?
![Page 19: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/19.jpg)
19
![Page 20: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/20.jpg)
20
![Page 21: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/21.jpg)
21
Balanced Quality Cost Budget
Intended Use Risk Assessment
COST
CAPABILITY FMEA
![Page 22: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/22.jpg)
22
How Much Error can I Tolerate? EXAMINE WHAT YOU TOLERATE…
vs
![Page 23: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/23.jpg)
23
Total Error Tolerated?
Accuracy—true/traceable to a standard? Do I always need this?
Bias/Systematic Error (SE)-why do different machines give
me different results?
Linearity – Define Limit Reportable range of results?
Specificity-measuring the quality I intended despite potential
interferences?
![Page 24: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/24.jpg)
24
RANDOM VARIATION Error Tolerated?
Precision: Random Error (RE) Repeatability/Robustness –
Sensitivity-Noise to Signal Ratio See through the static? Resolve small differences in quality despite assay
imprecision?
![Page 25: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/25.jpg)
25
Compare Tolerance of Error with
Measurement Capabilities
![Page 26: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/26.jpg)
26
Is my process in a state of statistical control? - Does it matter?
Quality Control Charts
![Page 27: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/27.jpg)
27
Control Charts – Levey Jennings
Defining Limits?? Mean, SD, %CV
Trends & Patterns
Shifts/Drifts of Quality
Reporting & Visualization Data
Shewart, Demming, Westgaurd (lab), etc….
![Page 28: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/28.jpg)
28
Common Cause vs Special Cause Variation
DEMMING SAYS: DON’T TAMPER
![Page 29: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/29.jpg)
29
Measurement Methodology of Clinical Trial Cells Data is limited –experience and publication minimal
Limited understanding of the process, what variables are really
affecting my quality?
How do you define quality in this new product? Who defines it?
Lacking manufacturer controls, manufacturer validated test kits,
established standards, clinical reagents, proficiency surveys, etc.
![Page 30: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/30.jpg)
30
Clinical Trial Cell Challenges… Is cell count/viability/dose always the most meaningful
indicator to predict therapeutic outcomes?
Do we always know what the true target cell marker is?
New indicators may be needed in order to measure the
most meaningful quality attributes
New Measurement Systems may need developed
Need Controls that meaningfully monitor variation in
assays-(i.e. frozen cell bank controls)
![Page 31: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/31.jpg)
31
Standardization Needs: Community peer
comparison initiatives?
Collaboration with Manufacturers &
Standardization Organizations for Support
![Page 32: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/32.jpg)
32
Develop Balanced Facility Quality Dashboard
Review Plan Optimize for Improvement Focus
Use SPC Understand Measurement Capabilities
Balance Investment in Capabilities according to Risk- Determined Tolerances
Seek Inter-laboratory Comparison Initiatives
Petition Manufacturer Development Support
![Page 33: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/33.jpg)
33
Questions
![Page 35: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/35.jpg)
35
Post Thaw Viability Scenario….
Facility began post thaw 7-AAD viability by Flow Cytometry for HPC, Apheresis products after a limited validation of 10 testing events (due to cost and low volume), all performed by one tech.
Upon implementation, noticed methods averaged 80% Viability post thaw as seen in validation, but rare outliers 50% post thaw viability with no manufacturing cause discovered during investigation. Engraftment outcomes were as expected.
Has this ever happened to you???
![Page 36: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/36.jpg)
36
Indicator Purpose:
Monitor stability of freeze/thaw manufacturing
process to detect clinically significant differences
in quality and functionality of cells.
WARNING FLAG for problems,
Evidence of acceptability/improvement during
process changes
![Page 37: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/37.jpg)
37
Having assay standardized to a gold
standard is not as important ….
The question is not….
Is the Viability of cells
as they are IV infused into the patient
really 50% or 80%?
![Page 38: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/38.jpg)
38
BUT is my assay capable to indicate
when the clinical quality of my cells
increases or decreases…
or is there too much “NOISE to hear”?
![Page 39: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/39.jpg)
39
To Be Meaningful Assay Must….
Exhibit a Large Signal
(sensitive to real changes in quality)
Have Small “Noise”
(comparably small amount of random error uncertainty and imprecision in the result)
![Page 40: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/40.jpg)
Desired Tolerance: +/- 10% Viability
Discovered
Assay NOT ROBUST—
Minor technique difference in tech preparation
vast differences in results
Poor Repeatability due to cell clumping
No meaningful QC available
Investigation annual process data: CV 40%
40
![Page 41: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/41.jpg)
IMPROVEMENT ACTION:
NOISE --Random Error
pipette measurement capability &
standardized rate diluent delivery electronic
positive displacement technology
Optimized “Thaw Diluent”: clumping
Coordinated 1 hour analysis time limit
Increased detail SOP
Performed training
41
![Page 42: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/42.jpg)
Next years processing data:
CV% Process decrease from 40 to 17%
Repeats gave near identical results
between operators.
Indicator now had value for intended use.
42
![Page 43: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/43.jpg)
43
*Do we need to transfuse
PLTs or NOT?
![Page 44: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/44.jpg)
Cancer center following engraftment of patients post
autologous HPC, Apheresis transplant. Patient CBCs
tested on small office Horiba Micros60 analyzer, but
sometimes sent to hospital analyzed on LH750.
Transplant nurses noticed significant discrepancies
occasionally (i.e. > 30 PLTS difference) between
machines.
QC was acceptable both instruments.
Manufacturer says they are both functioning within
spec??? 44
![Page 45: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/45.jpg)
45
Discrepant Results Monitoring patient
engraftment PLTs Transfusion Decisions?
Tolerance Limits
CLIA allows a tolerance limit of 25%
total error for PLT counts on hematology analyzers for
acceptable PT testing criteria as printed in the Federal Register Feb. 28, 1992; 57(40):
7002-186. Theses ranges are commonly used to define Analytical Quality Requirements.
![Page 46: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/46.jpg)
46
Tolerance Limits
However, for critical medical decisions,
biological cut-offs for 13% Total Allowable Error
have been defined in the literature for PLTs
(or +/- 2.5 at a true PLT value of 20). See Desirable Specification for Total Error, Imprecision, and Bias, derived from intra- and
inter-individual biologic variation and Current databases on biologic variation: pros, cons and
progress. Scand J Clin Lab Invest 1999; 59: 491-500. Ricos C, Alvarez V, et al.
Also,Clinical Decision Levels for laboratory Tests, Second Edition (Oradell NJ; Meical
Economics Books, 1987; Statland BE.)
![Page 47: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/47.jpg)
47
TRUE CAPABILITY ANALYZER HIDDEN:
Both Manufacturers State: Linearity down to 0.0 PLTs
PT testing in US and standard manufacturer QC does
NOT monitor precision/accuracy at critical PLT decision
points: 0-20 PLTs!
Most standard linearity kits provide limited testing at this
level and CLIA acceptability range is wide.
![Page 48: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/48.jpg)
48
UK NEQAS (United Kingdom national external Quality Assessment Scheme for Haematology)
General Haematology Study presented by Barbara De la Salle , 2009.
Thrombocytopenic Platelet Counting
Low Platelet Counting in EQA- the UK Experience
Discrepancies observed in automated Counting EQA: What do they mean?
![Page 49: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/49.jpg)
49
Using 13% TE cutoff…After Service!!!
![Page 50: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/50.jpg)
50
![Page 51: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/51.jpg)
51
# 0 1 2 3 4 5 7 8 9 10 112 9.3 20 23.1 37.6 95.7 134.1 191.2 227.8 622.8 1471 3730
3 10 18.5 24.3 37.2 95.5 139 190.1 232.9 605.7 1447 3731
4 9.7 19.4 23 38.4 95.1 134 201.9 229.2 615.6 1483 3506
5 9.3 19.7 22.9 38.8 97.3 136.5 193.2 237 610 1489 3726
Mean 9.58 19.40 23.33 38.00 95.90 135.90 194.10 231.73 613.53 1472.50 3673.25SD 0.34 0.648 0.66 0.73 0.97 2.37 5.36 4.12 7.39 18.57 111.52
CV% 3.55% 3.34% 2.81% 1.92% 1.01% 1.74% 2.76% 1.78% 1.21% 1.26% 3.04%
Expected
Results 9.705 19.41 23.1725 38.82 97.05 135.87 194.1 231.725 625.6575 1506.2125 3475.875
Range 7.2 - 12.2 16.91 - 21.91 20.67 - 25.67 33.82 - 43.82
Absolute Bias -0.13 -0.01 0.15 -0.82 -1.15 0.03 0.00 0.00 -12.13 -33.71 197.38% Bias -1.34% -0.05% 0.66% -2.11% -1.18% 0.02% 0.00% 0.00% -1.94% -2.24% 5.68%
Absolute
Accuracy
Limits 2.50 2.50 2.50 5.00 13.00 18.21 26.01 31.05 83.84 201.83 465.77
% Accuracy
Limits 25.76% 12.88% 10.79% 10.00% 13.40% 13.40% 13.40% 13.40% 13.40% 13.40% 13.40%
SD Limit @ 0
Bias 0.83 0.83 0.83 1.67 4.33 6.07 8.67 10.35 27.95 67.28 155.26
Acceptable? Passed Passed Passed Passed Passed Passed
Reference
Point:
Precision
Passed
Reference
Point:
Precision
Passed
Passed Passed Passed
Testing Performed by: Judy Charlton, MT, Hematology Specialist Date: 7/21/2011
Calculations & Graphical Analysis Performed by: Shadya George, MLS (ASCP)CM
Date: 7/25/2011
PLT Statistics Study: LH 750 AK06092
Total Error as %
-30%
-20%
-10%
0%
10%
20%
30%
0 500 1000 1500 2000 2500 3000 3500
Expected Result
% E
rro
r
% Bias
Upper Limit
Low er Limit
![Page 52: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/52.jpg)
52
![Page 53: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/53.jpg)
PLTs Ultra Low: Total Error as %
-40%
-30%
-20%
-10%
0%
10%
20%
30%
40%
0 10 20 30 40 50 60 70 80 90 100
Expected Result
% E
rro
r
% Bias
Upper Limit
Lower Limit
53
![Page 54: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/54.jpg)
54
• Ceased monitoring engraftment on Horiba Micros60
• Routine purchase of ultra-low PLT linearity kit for LH750
• Within 2 years increased instrument capability,
Replaced Hospital LH750 with DXH800
Replaced office Horiba Micros60 with Sysmex XS-1000i
![Page 55: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/55.jpg)
Will the real HCT please
stand up! 55
![Page 56: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/56.jpg)
*Transplant Center has been using the LH750 to
analyze HCT of cell therapy HPC, Apheresis
product for use in optimization of Apheresis
collection. The hospital replaced LH750 with the
advanced DXH800. Shortly after, Cancer Center
installed a new Sysmex XS-1000i to for better
precision of very low PLT count monitoring. The
Processing facility began a validation study for
comparison, but having problems with analyzer
agreement on HCT. Which one is “right”? 56
![Page 57: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/57.jpg)
57
Beckman Type Technology: Calculated HCT
HCT = MCV x RBCs MCHC=HGB x 100
10 HCT
Technology Review:
![Page 58: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/58.jpg)
58
Beckman Type Technology: Calculated HCT
Technology Disadvantages:
WBCs are counted in same chamber as RBCs.
High WBC /low RBC common in cell therapy
products interfere with all RBC indices
HCT is significantly falsely elevated in WBC
counts > 100 x103 cell/uL
![Page 59: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/59.jpg)
59
Beckman Type Technology: Calculated HCT Technology Disadvantages: Example Typical Interferences
![Page 60: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/60.jpg)
60
CALCULATED CORRECTION POSSIBLE LH750 & Similar Technologies:
Assuming 100% WBC interferences, acceptable correction can be
obtained mathematically. Improvement in use of corrected HCT
for optimization of collection, with adjustment in ranges.
• Use patient peripheral MCV, • Subtract WBC from RBC count • Manually Recalculate HCT
![Page 61: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/61.jpg)
61
Beckman Type Technology: Calculated HCT
DXH800 technology:
Auto-Correction of Interferences
Advances in algorithms developed for
abnormal peripheral bloods; less than
satisfactory for cell therapy products.
![Page 62: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/62.jpg)
62
DXH800 technology: Disadvantages: Manual calculated corrections impossible
due to differing degrees of failed “auto-correction”
technology. Significant unpredictable error, erratic results,
limited usefulness for collection optimization
![Page 63: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/63.jpg)
63
Manual Spun Hematocrit
Significant Interferences:
WBCs appeared trapped in the RBC fraction,
poor separation –
--falsely elevating the apparent HCT value.
Additional limitations: poor resolution, difficulty
reading due to large buffy coat, and trapped
plasma
![Page 64: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/64.jpg)
64
Sysmex-Direct Measured HCT
Based on the cumulative pulse heights of all
the RBCs counted as proportional to cell
volume. No calculation interferences
![Page 65: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/65.jpg)
65
HCT Counts: Technology Comparison
![Page 66: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/66.jpg)
LH750 #1 LH750 #2DXH800 #1
Manual Mode
DXH800 #1
Cassette
(2mL)
DX800#2
Manual
Mode
DX800#2
Cassette
(2mL)
Patient
Peripheral
Concurrent
MCV
LH750
Average
DXH800
Average
6 Ralph Tallent #2 11/9/2011 2.21 2.18 1.78 1.78 1.78 1.78 89.2 2.19 1.78
5 Ralph Tallent #3 11/9/2011 2.46 2.24 1.87 1.87 1.87 1.87 89.2 2.35 1.87
1 Donna Moore #2 11/7/2011 3.01 3.09 2.92 2.73 94.3 3.05 2.83
13 Wanda Miller #2 10/12/2011 3.35 3.10 2.99 106.8 3.35 3.04
2 Donna Moore #3 11/7/2011 3.48 3.33 2.92 2.83 94.3 3.41 2.88
14 Wanda Miller #3 10/12/2011 3.48 2.78 2.88 106.8 3.48 2.83
8 William Wells #3 11/14/2011 3.88 1.92 1.92 91.3 3.88 1.92
15 Wanda Miller #2 10/13/2011 4.27 3.80 3.80 105.5 4.27 3.80
3 Ralph Tallent #2 11/8/2011 4.52 4.24 3.89 3.80 4.15 88.4 4.52 4.02
17 Wanda Miller #2 10/14/2011 4.52 5.48 5.48 105.4 4.52 5.48
4 Ralph Tallent #3 11/8/2011 4.75 4.15 4.42 4.51 4.51 88.4 4.75 4.40
16 Wanda Miller #3 10/13/2011 5.25 5.24 4.43 4.64 105.5 5.25 4.54
7 William Wells #2 11/14/2011 5.50 2.47 2.56 91.3 5.50 2.51
18 Wanda Miller #3 10/14/2011 8.43 8.01 8.01 105.4 8.43 8.01
Average 4.22 3.22 3.27 3.92 3.04 3.62 4.21 3.57
Difference
r
LH750 vs DXH800
Calculated Corrected HCT : LH750 corrected using corrected RBC (Reported RBC - WBC) and patient concurrent MCV. DXH800 corrected
using patient concurrent MCV only (Instrument provides RBC auto correction in presence of high WBCs)
0.65
0.8517
DatePatient
Note: Wells sample contains Giant Plts flag which could interfere with the RBC count. DXH800 may be better at correcting
for this error, hence the two points that do not agree well. Without the well's points superior agreement.
Conclusion: DXH800 runs just slightly lower corrected HCT on average, clinically insignificant. Corrections equivalent.
#
66
HCT counts: Technology Comparison
HCT Comparison
y = 0.9172x - 0.2971
R2 = 0.7254
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
0.00 2.00 4.00 6.00 8.00 10.00
LH750
DX
H800
![Page 67: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/67.jpg)
67
WHICH ANALYZER IS REPORTING
THE “RIGHT” HCT?
Purpose: optimization of Apheresis collection to
limit granulocyte contamination and increase
efficiency collection.
![Page 68: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/68.jpg)
68
Again…
Having assay standardized to a gold
standard is not as important ….
The question is not
Is the HCT really 3, 2, 9 or 12%?
![Page 69: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/69.jpg)
69
BUT is my assay capable or SENSITIVE enough
for my intended use?
Increase and decrease proportionately to
the actual product HCT?
Specific admidst fluctuating interferences?
Can it be used to direct in apheresis
settings to improve collection?
![Page 70: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/70.jpg)
Outcome:
*LH750: Nurses report acceptable indicator use in setting
collection parameters using corrected values. However,
machine is gone.
*DXH800: Nurses report frustration and lack of
consistency in applying corrections based on product HCT.
Corrected and uncorrected HCT reported as not helpful.
*Recommended switching to HCT analysis using the
Sysmex XS-1000i, and re-setting new acceptable
expected ranges using statistical analysis.
70
![Page 71: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/71.jpg)
Stability Program
You prepare a comprehensive Stability Program. Your
program includes the cryopreservation of 6 vials to
be tested at: 30, 60, 90 days and 1, 3 years. One of
the products included in program was collected on
9/22/2015. You start acquiring data and have the
following results:
The first 4 specimens yielded the following results:
71
![Page 72: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/72.jpg)
Pre
Freeze TNCx1010 (Vol x WBC) =
7.10
7AAD (%
Alive)=
99.49
CFUx104/
kg=_____
Not
Done
CD34 %
=1.03
CD34x106/kg=
10.23
Post
Thaw WBC
(x106/ml)
TNCx1010
(Vol x WBC)
Total 7AAD
(% Alive)
CFU
(Growth) CD34%
CD34
Only
%Viabilit
y
Viable CD34 Dose
(x106/kg) (%
Recovery)
Tech Initials
30 days 276.0 7.73 52.48 Yes
No 1.54 68 7.81(43%) LC/MM`
60 days 290.0 8.12 49.55 Yes
No 2.74 92.7 13.78(76%) LC/RR
90 days 282.0 7.90 50.62 Yes
No 3.07 96.5 15.34(84%) LC/LCR
1 year 276.0 7.73 56.70 Yes
No 2.82 81.9 15.45(85) JM/YY
72
1. Is the difference in results due to different staff, different technique?
2. Should I check the gating technique used for all samples?
3. Should I trust these results? What this is really telling me?
![Page 73: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/73.jpg)
1. Is the difference in results due to different staff, different
technique?
*We decided to assign a team that does most of the post-thaw together. We
stated to get better results, more “credible”
2. Should I check the gating technique used for all samples?
*When we went back to check gating, we noticed we were all over in our
technique. With time, more cells (grans) will die and gating became more
challenging. We decided to “standardize” gating by using similar viability
gating for all vials. Now we checked results from previous thawing and
compare gating.
73
![Page 74: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/74.jpg)
74
ORIGINAL GATING RE- GATED
![Page 75: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/75.jpg)
75
ORIGINAL GATING RE- GATED
![Page 76: How to Apply Deming’s Philosophies in the World of …...SD, CV% 14 Adverse Product Infusion Reactions Annual Grade Internal Process Charted for each infusion on Activity resulting](https://reader034.vdocuments.us/reader034/viewer/2022042415/5f308533a15f1f322474b6c9/html5/thumbnails/76.jpg)
Pre
Freeze TNCx1010 (Vol x WBC) =
7.10
7AAD (%
Alive)=
99.49
CFUx104/
kg=_____
Not
Done
CD34 %
=1.03
CD34x106/kg=
10.23
Post
Thaw WBC
(x106/ml)
TNCx1010
(Vol x WBC)
Total 7AAD
(% Alive)
CFU
(Growth) CD34%
CD34
Only
%Viabili
ty
Viable CD34
Dose (x106/kg)
(% Recovery)
Tech Initials
30 days 296.0 6.22 90.1 Yes
No 0.93 93.4 5.74(103%)
60 days 300.0 6.30 88.8 Yes
No 1.01 96.1 6.23(117%)
90 days 288.5 6.06 90.6 Yes
No 0.92 96.4 5.56(104.7)
76
With standardization we started to see “better” results. One thing we noticed:
1. We continue to recover >100% of cells. Why?
2. Have you experience a change in methodology, ex. new cell counter with different technology,
and results for WBC from post-thaw are very different from counts made with previous instrument:
a. Did you stop the stability testing for the rest of the vials?