Department of Health and Human Services
Measuring Clinical Lab Ordering Quality: Theory and Practice
Measuring Clinical Lab Ordering Quality: Theory and PracticeSteven M. Asch MD MPH
VA, RAND, UCLAApril 29, 2005
INSTITUTE OF MEDICINEDEFINITION OF QUALITY The degree to which health services for individuals and populations * increase the likelihood of desired health outcomes and * are consistent with current professional knowledge
Lundberg , 1981
Were results used properly to improve care?
Has the right testbeen ordered?
Action
The 9 steps in the performance of any laboratorytest. The brain-to-brain turnaround time loop.
Interpretation
Reporting
Analysis
PreparationTransportation
Identification
Collection
Ordering
WHAT IS POOR QUALITY?WHAT IS POOR QUALITY?
• Too little care – underuse– Failure to provide an effective service when
it could have produced a good outcome
• Too much care – overuse– Providing care when its risks of harm
greater than potential benefit
• The wrong care – misuse– Avoidable complications of appropriate care
CONCEPTUAL FRAMEWORK
STRUCTURE PROCESS OUTCOMES
Technical Excellence
• Right choices
• Effective/skillful
Interpersonal Excellence
• Patient-centered
• Responsive
Functional Status
Satisfaction
Mortality
Biological Status
Health CareOrganization
Characteristics
ProviderCharacteristics
CommunityCharacteristics
PopulationCharacteristics
EXAMPLES OF STRUCTURAL MEASURESEXAMPLES OF STRUCTURAL MEASURES
• Health care organization characteristics- Weekend and night hours and
convenient locations of laboratories- Volume
• Provider characteristics– Number of pathologists– Training of laboratory staff
CONCEPTUAL FRAMEWORK
STRUCTURE PROCESS OUTCOMES
Technical Excellence
• Right choices
• Effective/skillful
Interpersonal Excellence
• Patient-centered
• Responsive
Functional Status
Satisfaction
Mortality
Biological Status
Health CareOrganization
Characteristics
ProviderCharacteristics
CommunityCharacteristics
PopulationCharacteristics
4%
30%33%
29%33%
86%
60%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
All U/A Glu K Cr Chol Trig
% Adherence
HTN NEW DIAGNOSIS LABSHTN NEW DIAGNOSIS LABS
Asch et. al. BMC CV, 2005
QATOOL SCORES BY MODEQATOOL SCORES BY MODE
Visit 73%
Medication 69%
Immunization 66%
Physical Exam 63%
Laboratory/Radiology 62%
Surgery 57%
History 43%
Education 18%McGlynn, Asch et. al. NEJM 2003
CONCEPTUAL FRAMEWORK
STRUCTURE PROCESS OUTCOMES
Technical Excellence
• Right choices
• Effective/skillful
Interpersonal Excellence
• Patient-centered
• Responsive
Functional Status
Satisfaction
Mortality
Biological Status
Health CareOrganization
Characteristics
ProviderCharacteristics
CommunityCharacteristics
PopulationCharacteristics
WHY MEASURE OUTCOMES?WHY MEASURE OUTCOMES?
– Allow innovation in process
– People care about outcomes directly
SAMPLE SIZE PROBLEMSSAMPLE SIZE PROBLEMS
– For mortality, need huge samples:• CHF patients: 12% vs 16%, need 957
patients at each hospital.
– Rarer outcomes• People care, but statistical comparison is
impossible.
DOES SICKNESS OR QUALITY DETERMINE CHF MORTALITY?Sickness at Process
Admission Poor Medium Good Total
Least Sick 1/4 4 7 4 5Middle 1/2 11 8 8 9Most Sick 1/4 37 32 26 32Total 16 14 12 14
ACCOUNTABILITY: IS PROVIDER RESPONSIBLE FOR PROBLEM?
ACCOUNTABILITY: IS PROVIDER RESPONSIBLE FOR PROBLEM?
– Current treatment must have big impact relative to other factors.
– Do not want providers avoiding those who:• have a bigger chance of problems• are less likely to adhere to treatment
CHOOSING MEASURES:PRACTICAL CONSIDERATIONS
– Choosing areas to measure
– Selecting indicators
– Designing specifications
– Testing the measure
CHOOSING AREAS:ASSESSING HEALTH IMPORTANCE
– Mortality
– Morbidity
– Utilization
– Cost
PREVALENCE OF SELECTED ACUTE CONDITIONS AMONG WORKING ADULTS
Condition Work Loss Days/100 Persons
Injuries
Influenza
Infections and parasitic disease
Common cold
Digestive system conditions
Other upper respiratory
Acute ear infections
85.5
53.1
20.6
15.4
12.3
9.3
3.2
CHOOSING AREAS: POTENTIAL FOR IMPROVEMENT
CHOOSING AREAS: POTENTIAL FOR IMPROVEMENT
– What are the key outcomes of interest?
– What processes produce those outcomes?
– How well are key elements of care delivered today?
– How variable is care delivery?
CHOOSING MEASURES: DEGREE OF PROVIDER CONTROL
CHOOSING MEASURES: DEGREE OF PROVIDER CONTROL
– How might the measure be affected by characteristics of the enrolled population?
– What actions can providers or clinical laboratories take to improve performance?
STRENGTH OF SCIENTIFIC EVIDENCESTRENGTH OF SCIENTIFIC EVIDENCE
I: Randomized controlled trialII-1: Nonrandomized controlled trialI-2: Cohort or case control studiesII-3: Multiple time seriesIII: Opinions or descriptive studies
COST-EFFECTIVENESS OF PROCESS
MMR Immunization $14 saved/$1 spent
Cervical cancer screening $21,000 spent/year (ages 20-28)
Cervical cancer screening $11,000 spent/year (ages 29-50)
DESIGNING MEASURE SPECIFICATIONSDESIGNING MEASURE SPECIFICATIONS
– Define indicator– Identify target population– Define eligible population– Determine need for risk adjustment– Identify data sources– Write data collection instructions– Develop scoring rules
Example measureExample measure
• Men with a new diagnosis of prostate cancer, who have not had a serum PSA in the prior three months, should have serum PSA checked within one month after diagnosis or prior to any treatment, whichever comes first.
EVALUATING DATA SOURCES
DATA SOURCE STRENGTHS WEAKNESSES
Medical Record Clinical Detail Expense Missing links
Administrative Use of services Clinical detail
Patient Surveys General Health Expense
Interpersonal Clinical detail
TESTING THE MEASURETESTING THE MEASURE
– Reliability: The proportion of times that repeated use of measure in same population gives the same result
– Validity: The extent to which the measure accurately represents the concept being assessed
– Interpretability: Ease with which target audience can understand and use information generated by measure
WHY SHOULD CLINICIANS CARE ABOUT MEASURING QUALITY?
WHY SHOULD CLINICIANS CARE ABOUT MEASURING QUALITY?
– Internal quality improvement
– External monitoring and evaluation
– Consumer/purchaser decision-making
ADEQUACY OF CASE-MIX CONTROLADEQUACY OF CASE-MIX CONTROL
– Severity of disease– Incidence and prevalence by
demographics•age•race•gender