adapting ahrq patient safety indicators to qio data jocelyn andrel, msph charles p. schade, md, mph...
TRANSCRIPT
Adapting AHRQ Patient Safety Indicators to QIO Data
Jocelyn Andrel, MSPHCharles P. Schade, MD, MPHPatricia Ruddick, RN, MSN
Outline of Presentation
What are AHRQ Patient Safety Indicators?How can you use QIO data to get them?What are their characteristics in one state?How can you share them with hospitals?What do one state’s hospitals think of
them?How do they relate to other evidence about
safety in a state’s hospitals?
AHRQ Patient Safety Indicators
What they areHow to compute them
AHRQ Patient Safety Indicators: Background
Early 1990sDeveloped by the Agency for Healthcare
Research and Quality (AHRQ) to measure the safety of hospital care using administrative inpatient discharge data. The Indicators screen for problems that patients experience as a result of exposure to the healthcare system.
Concept of PSIs
Based on conditions that clearly reflect medical error (foreign body left in)
Based on conditions that could reflect medical error (PE or DVT)
Not based on underlying comorbidities
Steps to determine PSIs
1. Define the concepts and the evaluation framework
2. Search the literature to identify potential PSIs
3. Develop a candidate list of PSIs
4. Review the PSIs
5. Evaluate the PSIs using empirical analysis
Limitations
Some events don’t show up in discharge data– Adverse drug reactions– Medical events– Psychiatric events.
Administrative data may not address finer detail Patient Safety Indicators should be used to prompt
investigation into areas where the hospital could potentially improve quality of care
PSIs
Accidental puncture or laceration Complications of Anesthesia Death in low mortality DRGs Decubitus Ulcer Failure to Rescue Foreign body left in during procedure Iatrogenic pneumothorax Postoperative hemorrhage or hematoma Postoperative hip fracture
PSIs continued
Postoperative physiologic and metabolic derangement Postoperative pulmonary embolism or DVT Postoperative respiratory failure Postoperative sepsis Postoperative wound dehiscence Selected infections due to medical care Transfusion reaction Plus 4 Obstetric measures not addressed here
Converting ISAT data
General Instructions from AHRQ– 1. The data must be in SAS– 2. You may have to recode specific data elements
to match what is used in the software.Fortunately, conversion of the ISAT file to
comport with the AHRQ input requirements is fairly simple
Conversion Elements
Creating/Formatting Variables– Age– Length of Stay– Create variables for the number of diagnoses and the
number of procedures– Set payor to the code for Medicare– Format Hospital codes, Race, Sex, Key, Hospital ID,
DRG, Admission Source, Admission Type – Rename Diagnosis and Procedure codes
Major Diagnostic Codes from the HSE Claims Lookup Table
And then…
The ISAT file is ready to be input into the AHRQ Patient Safety Indicator programs
AHRQ Patient Safety Indicators:Results in a Single State
Methods
Adapted standard output (psp3 table at hospital level) to a graphic display and comparative report
Generated histograms of hospital performance on each indicator for 2000-2002
Generalized code to run with any state’s data as input
Results: Distribution of Hospitals
Some indicators appeared normally distributed
Some were highly skewed, with outliersSome appeared bimodal
AHRQ Risk Adjusted PSI RateFailure to RescueWV Hospitals, 2002
0
5
10
15
20
Nu
mb
er
of
Ho
sp
ita
ls
0.0
to
25
.0
25
.0 t
o 7
5.0
75
.0 t
o 1
25
.0
12
5.0
to
17
5.0
17
5.0
to
22
5.0
22
5.0
to
27
5.0
27
5.0
to
32
5.0
32
5.0
to
37
5.0
37
5.0
to
42
5.0
42
5.0
to
47
5.0
47
5.0
or
mo
re
Rate per 1,000 cases
Death rate in dischargeswith potential complicationsof care, e.g., pneumonia, DVT/PE, sepsis, acute renal failure, shock/cardiac arrest, GI hemorrhage/acute ulcer.
Definition
AHRQ Risk Adjusted PSI RateSelected Infections Due To Medical CareWV Hospitals, 2002
0
5
10
15
20
Nu
mb
er
of
Ho
sp
ita
ls
0.0
to .
5
.5 t
o 1.
5
1.5
to 2
.5
2.5
to 3
.5
3.5
to 4
.5
4.5
to 5
.5
5.5
to 6
.5
6.5
to 7
.5
7.5
to 8
.5
8.5
to 9
.5
9.5
or m
ore
Rate per 1,000 cases
Discharges with ICD-9-CMcode of 999.3 or 996.62 in anysecondary diagnosis fieldexcluding immunocompro-mised and cancer
Definition
AHRQ Risk Adjusted PSI RatePost-Operative SepsisWV Hospitals, 2002
0
5
10
15
20
Nu
mb
er
of
Ho
sp
ita
ls
0.0
to
2.5
2.5
to
7.5
7.5
to
12
.5
12
.5 t
o 1
7.5
17
.5 t
o 2
2.5
22
.5 t
o 2
7.5
27
.5 t
o 3
2.5
32
.5 t
o 3
7.5
37
.5 t
o 4
2.5
42
.5 t
o 4
7.5
47
.5 o
r m
ore
Rate per 1,000 cases
Elective surgical dischargeswith ICD-9-CM code for sepsis in any secondarydiagnosis field excludingimmunocompromised andcancer
Definition
Results: Statewide Values Over 3 Years
We also used the following format for the tabular report to individual hospitals
Most indicators based on small numerators statewide and appeared to show statistical fluctuation from year to year
Failure to rescue declining?Postop sepsis and DVT/PE increasing?
AHRQ Risk-Adjusted Patient Safety Indicator Rates--WV, 2000Statewide
Number Rate* Hospital Percentiles*Num-erator
Denom-inator
Crude Adjusted 10th 50th 90th
Complications of Anesthesia 11 29,053 0.4 0.3 0 0 1.1Death in Low Mortality DRGs 52 13,278 3.9 1.9 0 0.099 5.2Decubitus Ulcer 1,670 57,736 28.9 25.0 4.3 24.2 49.0Failure to Rescue 741 4,656 159.1 176.2 54.6 168.5 240.7Foreign Body Left in During Proc 6 131,145 0.046 0.063 0.013 0.027 0.222Iatrogenic Pneumothorax 124 119,616 1.0 0.9 0 0.448 2.1Infection Due to Medical Care 230 110,892 2.1 1.7 0 0.897 2.8Postoperative Hip Fracture 52 21,083 2.5 1.8 0 0.724 10.0Postop Hemor or Hemat-No Prday 62 29,028 2.1 1.9 0 0.570 4.3Postop Physio Metabol Derangmnt 21 12,576 1.7 1.1 0 0.397 6.0Postop Respiratory Failure 62 9,278 6.7 2.6 0 1.3 9.3Postop PE or DVT-No Prday 330 28,870 11.4 8.7 0 4.2 14.9Postoperative Sepsis 47 4,946 9.5 7.9 0 5.0 21.5Postoperative Wound Dehiscence 20 5,399 3.7 2.4 0 0 8.5Accidental Puncture/Laceration 290 131,049 2.2 3.3 1.7 2.6 3.9
*Rates per 1,000 eligible cases
Indicator
AHRQ Risk-Adjusted Patient Safety Indicator Rates--WV, 2001Statewide
Number Rate* Hospital Percentiles*Num-erator
Denom-inator
Crude Adjusted 10th 50th 90th
Complications of Anesthesia 7 28,926 0.242 0.185 0 0 0.185Death in Low Mortality DRGs 47 13,657 3.4 1.6 0 0 5.5Decubitus Ulcer 1,552 56,299 27.6 24.0 9.7 22.0 42.8Failure to Rescue 692 5,209 132.8 151.7 76.5 160.1 232.2Foreign Body Left in During Proc 5 129,442 0.039 0.056 0.008 0.026 0.131Iatrogenic Pneumothorax 119 117,966 1.0 0.9 0.044 0.281 1.5Infection Due to Medical Care 271 109,233 2.5 2.1 0.308 1.4 3.2Postoperative Hip Fracture 42 20,674 2.0 1.3 0 0.670 7.0Postop Hemor or Hemat-No Prday 60 28,876 2.1 1.9 0 0.855 5.3Postop Physio Metabol Derangmnt 15 12,971 1.2 0.6 0 0.138 1.3Postop Respiratory Failure 82 9,542 8.6 4.2 0 0 13.2Postop PE or DVT-No Prday 360 28,689 12.5 10.1 0 8.8 17.8Postoperative Sepsis 48 4,990 9.6 7.8 0 6.6 23.3Postoperative Wound Dehiscence 22 5,367 4.1 2.9 0 0 9.0Accidental Puncture/Laceration 272 129,303 2.1 3.2 1.5 2.8 4.0
*Rates per 1,000 eligible cases
Indicator
AHRQ Risk-Adjusted Patient Safety Indicator Rates--WV, 2002Statewide
Number Rate* Hospital Percentiles*Num-erator
Denom-inator
Crude Adjusted 10th 50th 90th
Complications of Anesthesia 11 26,188 0.420 0.347 0 0 0.347Death in Low Mortality DRGs 59 13,174 4.5 2.6 0 1.4 8.5Decubitus Ulcer 1,604 53,879 29.8 26.1 4.3 25.9 36.5Failure to Rescue 679 5,031 135.0 148.4 72.5 159.0 225.1Foreign Body Left in During Proc 4 124,902 0.032 0.050 0.008 0.026 0.050Iatrogenic Pneumothorax 100 114,155 0.9 0.8 0 0.639 1.4Infection Due to Medical Care 241 105,239 2.3 1.9 0.021 1.2 3.2Postoperative Hip Fracture 51 17,965 2.8 2.1 0 0 6.9Postop Hemor or Hemat-No Prday 64 26,155 2.4 2.2 0 0.610 3.8Postop Physio Metabol Derangmnt 29 11,993 2.4 1.8 0 0.147 3.1Postop Respiratory Failure 82 9,564 8.6 4.4 0 1.5 27.3Postop PE or DVT-No Prday 381 25,961 14.7 11.4 0 8.7 21.4Postoperative Sepsis 57 4,689 12.2 10.3 0 5.7 36.0Postoperative Wound Dehiscence 17 5,091 3.3 2.3 0 0 14.4Accidental Puncture/Laceration 219 124,798 1.8 3.0 1.5 2.6 3.7
*Rates per 1,000 eligible cases
Indicator
Report to Hospitals
Calendar year 2002, with offer of other years’ results
Tabular (see previous) and graphical formatExplanatory letter, definitions of indicatorsMailed to hospital patient safety contact or
HCQIP contactAsked for feedback on report contents and
utility
AHRQ Patient Safety Indicators--WV, 2002 (Risk Adjusted)Hospital A
Medicare Fee For Service Patients
0.001 0.01 0.1 1 10 100 1000
Indicator Rate (Per Thousand Cases)
(0.544)Complications of Anesthesia
(2.7)Death in Low Mortality DRGs
(25.7)Decubitus Ulcer
(160.6)Failure to Rescue
(0.018)Foreign Body Left in During Proc
(0.859)Iatrogenic Pneumothorax
(1.5)Infection Due to Medical Care
(3.8)Postoperative Hip Fracture
(2.3)Postop Hemor or Hemat-No Prday
(1.1)Postop Physio Metabol Derangmnt
(17.4)Postop Respiratory Failure
(10.0)Postop PE or DVT-No Prday
(16.4)Postoperative Sepsis
(0)Postoperative Wound Dehiscence
(2.1)Accidental Puncture/Laceration
State Range (10th-90th Percentile)
State Median (50th Percentile)
Hospital Risk Adjusted Value
Patient Safety Indicators:
Implications for WVMI’s Patient Safety Project
Specific Goals of the WV Patient Safety Project
Establish a system of confidential reporting for medical errors and near misses
Stimulate reporting of such events by developing a non-punitive response system
Provide feedback of surveillance data at appropriate levels of aggregation
Educate consumers of healthcare about patient safety guidelines
Comparing PSI Data to the Patient Safety Data Purpose:1. Ascertain the usefulness of the PSI
data in hospitals in West Virginia2. Compare the data received from the
PSI data to the data received from the Patient Safety Project
3. Explore further opportunities for quality improvement projects
PSI/Patient Safety Data Study
CEOs and Quality Improvement staff from 41 acute care West Virginia hospitals received:
1. Information letter2. Patient Safety Indicator definitions3. Table which showed the actual number of specific
incidences of each PSI (2002), crude and adjusted rates, and comparative percentiles of all hospitals in the state combined
4. Graphical representation of the data presented in the table
5. Brief questionnaire on the usefulness of the graph and tables
Patient Safety Questionnaire
Feedback on the Patient Safety Indicator Reports
Please take a minute or two to tell us your reaction to the enclosed reports. Your responses will be kept confidential and used only for evaluating this project.
1. Please check the box that most closely describes your role in the hospital
Quality improvement staff
Patient safety staff
Medical staff
Clinical nursing staff
Administration
Other ________________
2. Please circle the number indicating the extent to which you agree or disagree with each statement, where:
5 = strongly agree4 = agree3 = indifferent2 = disagree1 = strongly disagreeIf a question is not applicable to your situation, please leave
it blank. Strongly agree...strongly disagree a. The patient safety indicator reports were easy to understand
5 4 3 2 1 b. The graphic report was easier to use than the tabular report
5 4 3 2 1
Patient Safety Questionnaire, cont.
c. The tabular report provided more information than the graphic report
5 4 3 2 1 d. My hospital’s indicator results, compared with the state’s rates,
are about what I would have expected5 4 3 2 1
e. I want to share the report with colleagues in my hospital
5 4 3 2 1 f. I need additional information about one or more of the
indicators 5 4 3 2 1
Patient Safety Questionnaire, cont.
3. Please tell us anything you liked about the reports:
4. Please let us know of anything you did not like about the reports:
5. Finally, please let us know any questions you’d like answered about the reports:
Patient Safety Questionnaire, cont.
Results(14/41 questionnaires returned)
5 4 3 2 1a. The PSI reports were easy to understand 28% 42% 14% 14%
b. The graphic report was easier to use than the tabular report
35% 28% 14% 14% 7%
c. The tabular report provided more information than the graphic report
21% 28% 35% 7% 14%
d. My hospital’s indicator results, compared with the state’s rates, are about what I expected
25% 25% 33% 17%
e. I want to share the report with colleagues in my hospital
35% 28% 21% 7% 7%
f. I need additional information about one or more of the indicators
50% 7% 14% 21% 7%
Likes/dislikes about the PSI Reports
Likes: Good overview of our results Serves as a step for further analysis Great idea-shared this with Department of
Medicine Graphs were self-explanatoryDislikes: Leaves many questions unanswered Need more current information Would like to set up and run on their own
ExamplesPatient Safety Indicator Patient Safety Event
Complications of anesthesia Adverse Clinical Event
-sedation management
-complication/monitoring
Decubitus ulcers Adverse Clinical Event
-skin integrity
-decubitus
Foreign body left in during procedure
Adverse Clinical Event
-operative/invasive procedure
-instrument/needle/sponge count
Infection due to medical care Adverse Clinical Event
-infection
May have to search several fields to find
coordinating PSI
Conclusions Data captured from PSIs may best be used to investigate
potential patient safety problems when hospitals compare PSIs to the coordinating medical error on the incident reporting tool since:
Some events don’t show up in discharge data that are captured in the incident reporting tool, e.g.
– Adverse drug reactions– Administrative events – Fall events – Employee events– Visitor events
PSI data is more general and may have to use several fields in the incident reporting tool to capture complete PSI data
WVMI plans to:
Compare 2003 PSI data with data from Web-based incident reporting tool for hospitals that are part of the WV Patient Safety Project
– Unable to compare 2002 PSI data with patient safety data since the hospitals participating in the Patient Safety Project did not start until middle of 2002; and indicators do not correspond exactly.
– Provide this information to each participating hospital in order that they will be able to compare their reporting rates to PSI data
Source
AHRQ patient safety indicator programs http://www.qualityindicators.ahrq.gov
Conversion routines and hospital output code:
– [email protected]– [email protected]
WVMI’s Patient Safety Project– [email protected]