part 1 – building blocks - qualis...
TRANSCRIPT
NHSN Analytics For real (and busy!) people
Jamie Moran, MSN, RN, CMSRN, CIC Martha Jaworski, MS,RN, CIC
May 29, 2015
Part 1 – Building Blocks
2
Qualis Health • A leading national population health
management organization • The Medicare Quality Innovation Network - Quality
Improvement Organization (QIN-QIO) for Idaho and Washington
The QIO Program • One of the largest federal programs dedicated to
improving health quality at the local level
3
To understand God's thoughts we must study statistics, for these are the measure of His purpose.
Florence Nightingale
4
The manipulation of statistical formulas is no substitute for knowing what one is doing.
Hubert M. Blalock, Jr., Social Statistics
5
6
7
8
Before NHSN… • Infection Rates – Examples of Variation
• Numerator • Criteria…no criteria…physician diagnosis • Number of total infections in a given period • Number of specific infections in a given period
• Denominator • Per admission • Per patient day • Per device use • Per device day
• Scale • Per 1 • Per 100 (expressed as a percent) • Per 1000
9
After NHSN…
• Standardization in surveillance definitions
• Common denominators • Common scales • Risk-adjusted benchmarking • PURPOSE: To improve!
10
Standardized rates…
11
12
Infection Rates
13
NHSN Published Rates…
14
15
16
17
18
19
Benchmarking…
• For CAUTIs and CLABSIs: • Incidence Density Rate (IDR)
• (# of Infections ÷ Device Days) * 1000
• EXAMPLE… • Numerator – 1 CLABSI • Denominator – 781 CL days • IDR = 1.28 per 1000 line days
20
21
How many infections should you expect?
22
Expected Number of Infections
Depends on the infection type… • For CAUTIs and CLABSIs:
• Pooled Means • Combined rate of all reporting hospitals…
(Pooled Mean ÷1000)* Your Device Days
23
24
Expected Number of Infections
Depends on the infection type… • For CAUTIs and CLABSIs:
• Pooled Means • Combined rate of all reporting hospitals…
(Pooled Mean ÷1000)* Your Device Days
• (0.8 ÷1000)* 781 CL Days = 0.62
• You can expect 0.62 CLABSIs in this period
• If you want to aim for the 10th or 25th percentiles, use those benchmarks in the formula (though answer may be zero!)
25
Risk Adjustments for Expected Numbers
26
Expected Number of Infections Surgical Site Infections
This is the model reported to CMS!
27
The Standardized Infection Ratio (SIR)
27
28
Standardized Infection Ratio (SIR)
• At its simplest: Number of Infections
Expected Number of Infections
• This type of ratio is commonly referred to as “observed over expected”
• If the observed infections = the number expected, then the ratio is 1.0
• The ratio is risk-adjusted in the Expected Number
29
http://www.cdc.gov/nhsn/PDFs/sir/RatesSIRsReference_Jan2015.pdf
2013 Summary Data
2006-2008 Summary Data
2009 Summary Data
Why? To measure performance over time against a static baseline using the “SIR”
2006-2008 UNPUBLISHED Summary Data
Pay attention to source of expected number!
30
Standardized Infection Ratio (SIR)
• Examples… • SIR = Observed Expected • M/S ICU, 24 Beds • Pooled Mean = 0.8 • Line Days = 2250 • # CLABSIs = 1 • Expected Number = ? 1.8
(Pooled Mean/1000)*Device Days
1 . 1.8
SIR = 0.556
31
Standardized Infection Ratio (SIR)
• Examples… • SIR = Observed Expected • M/S ICU, 24 Beds • Pooled Mean = 0.8 • Line Days = 2250 • # CLABSIs = 2 • Expected Number = 1.8 (Pooled Mean/1000)*Device Days)
2 . 1.8
SIR = 1.11
32
Standardized Infection Ratio (SIR) • Benefits…
• Unlike rates, SIR can be aggregated with validity by… • Unit Locations • Acuity Levels • Facilities • Groups • States • Nation • Time
• Sum of observed infections divided by sum of expected infections
33
Standardized Infection Ratio (SIR) • Limitations…
• If an Expected Number is less than 1, a SIR is not calculated by NHSN
• 40-bed hospital has an Expected Number of CLABSIs of 0.5 for a one year period
• But they had 1 CLABSI
• SIR would calculate as 2.0
• SIR cannot be interpreted correctly when Expected Number is less than 1
• You’ll see “ ” in reports when Expected Number is less than 1
• Other data may be excluded from the SIR, depending on circumstance
34
Standardized Infection Ratio (SIR)
35
Standardized Infection Ratio (SIR) • Uses…
• CMS reporting for Inpatient Quality Reporting Program • Benchmark for Value-Based Purchasing • Benchmark for Hospital-Acquired Conditions Penalty Program
• National Action Plan to Prevent Health Care-Associated Infections: Roadmap to Elimination
• CDC National and State HAI Progress Reports
36
In-Plan SIRs vs. CMS-Reported SIRs
Beginning January 1, 2015
37
Statistical Significance
37
38
Statistical Significance
• “Statistical significance…” • Don’t be scared! • It’s just a measure of reliability.
• Attempts to answer these questions: • For SIRs
• Is the SIR really different from 1.0 ?
• For Rates • Is the rate really different from the pooled mean?
39
Statistical Significance p-Value • “p” is the probability that the difference or relationship occurred by
chance
• “Null hypothesis” (H0) is the assumption that there is no real difference between your value and the benchmark
• α (alpha) is the amount of error you’re willing to accept in rejecting the H0. NHSN uses α = 0.05 to reject H0
• Therefore, a p-Value < 0.05 is considered “significant”
• In other words, when p < 0.05, you can be 95% confident the difference between your statistic and the benchmark is reliable
• NHSN calculates the p-Value using a chi-square (χ2)-like test for SIRs and a proportions test for rates
40
Statistical Significance Confidence Intervals (CI) • A confidence interval (CI) provides us with a lower and upper limit of
values between which the SIR could fall if the entire population was measured.
• If 1.0 falls within the CI, the results are not statistically significant.
• 95% CIs are very sensitive to sample size; the smaller the size, the wider the interval
SIR 1.8
UL 2.1
LL 0.8
0.0
1.0
41
Statistical Significance
SIR 1.8
UL 2.9
LL 1.6
0.0
1.0
Confidence Intervals (CI) • A confidence interval (CI) provides us with a lower and upper limit of
values between which the SIR could fall if the entire population was measured.
• If 1.0 falls within the CI, the results are not statistically significant.
• 95% CIs are very sensitive to sample size; the smaller the size, the wider the interval
42
NHSN Statistics Calculator
43
Statistical Significance
• So what? • p-values and 95%-CIs are a measure of
reliability • Significance is rare, but when it is…pay
attention! • The significance means there is a true difference
between the two values • Has something changed? • Does something need your attention? • Have your improvement efforts paid off?
44
TAP Strategy and the CAD
45
CDC’s Innovation…
TAP Reports!
46
47
Goals…
http://www.hsdl.org/?view&did=739815
48
TAP Reports
• New statistic… “C A D” • Cumulative Attributable Difference
• May be a positive number (“excessive”) • May be a negative number (“prevented”)
49
TAP Reports
• Example #1 • 11 HO CDI • 9.354 Expected • HHS Target: 0.70
CAD = 11 – (9.354 * 0.70) CAD = 4.452 (positive)
• Example #2 • 3 HO CDI • 9.354 Expected • HHS Target: 0.70
CAD = 3 – (9.354 * 0.70) CAD = -3.548 (negative)
Note that the effect of the CAD is to lower the “Expected Number” by a given percentage to reach the HHS goal. (In this case, from 9.354 to 6.548)
50
TAP Reports
• Provide the facility-level CAD, by HAI type • Sum of all infections – (sum of all expected * HHS Target)
• Rank-order units (highest CAD first) within a facility
• Allows facilities to “Target” highest contributors to the overall facility CAD
51
numPathUTI: Number and pathogens isolated in the infections # (EC, YS, PA, KS, PM, ES)
EC = E. coli YS = yeast PA = P. aeruginosa KS = K. pneumoniae/oxytoca PM = Proteus mirabilis ES = Enterococcus species
52
53
With TAP Reports….
54
Q & A
55
For survey:
www.SurveyMonkey.com/s/VD3R9K9
For more information: www.Medicare.QualisHealth.org/projects/healthcare-
associated-infection
This material was prepared by Qualis Health, the Medicare Quality Innovation Network - Quality Improvement Organization (QIN-QIO) for Idaho and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C1-QH-1766-05-15
Contact Martha Jaworski, RN, MS, CIC
Idaho QI Consultant [email protected]
208-383-5944
Jamie Moran, MSN, RN, CIC Washington QI Consultant [email protected]
206-288-2512