part 1 – building blocks - qualis...

NHSN Analytics For real (and busy!) people

Jamie Moran, MSN, RN, CMSRN, CIC Martha Jaworski, MS,RN, CIC

May 29, 2015

Part 1 – Building Blocks

2

Qualis Health • A leading national population health

management organization • The Medicare Quality Innovation Network - Quality

Improvement Organization (QIN-QIO) for Idaho and Washington

The QIO Program • One of the largest federal programs dedicated to

improving health quality at the local level

3

To understand God's thoughts we must study statistics, for these are the measure of His purpose.

Florence Nightingale

4

The manipulation of statistical formulas is no substitute for knowing what one is doing.

Hubert M. Blalock, Jr., Social Statistics

5

6

7

8

Before NHSN… • Infection Rates – Examples of Variation

• Numerator • Criteria…no criteria…physician diagnosis • Number of total infections in a given period • Number of specific infections in a given period

• Denominator • Per admission • Per patient day • Per device use • Per device day

• Scale • Per 1 • Per 100 (expressed as a percent) • Per 1000

9

After NHSN…

• Standardization in surveillance definitions

• Common denominators • Common scales • Risk-adjusted benchmarking • PURPOSE: To improve!

10

Standardized rates…

11

12

Infection Rates

13

NHSN Published Rates…

14

15

16

17

18

19

Benchmarking…

• For CAUTIs and CLABSIs: • Incidence Density Rate (IDR)

• (# of Infections ÷ Device Days) * 1000

• EXAMPLE… • Numerator – 1 CLABSI • Denominator – 781 CL days • IDR = 1.28 per 1000 line days

20

21

How many infections should you expect?

22

Expected Number of Infections

Depends on the infection type… • For CAUTIs and CLABSIs:

• Pooled Means • Combined rate of all reporting hospitals…

(Pooled Mean ÷1000)* Your Device Days

23

24

Expected Number of Infections

Depends on the infection type… • For CAUTIs and CLABSIs:

• Pooled Means • Combined rate of all reporting hospitals…

(Pooled Mean ÷1000)* Your Device Days

• (0.8 ÷1000)* 781 CL Days = 0.62

• You can expect 0.62 CLABSIs in this period

• If you want to aim for the 10th or 25th percentiles, use those benchmarks in the formula (though answer may be zero!)

25

Risk Adjustments for Expected Numbers

26

Expected Number of Infections Surgical Site Infections

This is the model reported to CMS!

27

The Standardized Infection Ratio (SIR)

27

28

Standardized Infection Ratio (SIR)

• At its simplest: Number of Infections

Expected Number of Infections

• This type of ratio is commonly referred to as “observed over expected”

• If the observed infections = the number expected, then the ratio is 1.0

• The ratio is risk-adjusted in the Expected Number

29

http://www.cdc.gov/nhsn/PDFs/sir/RatesSIRsReference_Jan2015.pdf

2013 Summary Data

2006-2008 Summary Data

2009 Summary Data

Why? To measure performance over time against a static baseline using the “SIR”

2006-2008 UNPUBLISHED Summary Data

Pay attention to source of expected number!

30

Standardized Infection Ratio (SIR)

• Examples… • SIR = Observed Expected • M/S ICU, 24 Beds • Pooled Mean = 0.8 • Line Days = 2250 • # CLABSIs = 1 • Expected Number = ? 1.8

(Pooled Mean/1000)*Device Days

1 . 1.8

SIR = 0.556

31

Standardized Infection Ratio (SIR)

• Examples… • SIR = Observed Expected • M/S ICU, 24 Beds • Pooled Mean = 0.8 • Line Days = 2250 • # CLABSIs = 2 • Expected Number = 1.8 (Pooled Mean/1000)*Device Days)

2 . 1.8

SIR = 1.11

32

Standardized Infection Ratio (SIR) • Benefits…

• Unlike rates, SIR can be aggregated with validity by… • Unit Locations • Acuity Levels • Facilities • Groups • States • Nation • Time

• Sum of observed infections divided by sum of expected infections

33

Standardized Infection Ratio (SIR) • Limitations…

• If an Expected Number is less than 1, a SIR is not calculated by NHSN

• 40-bed hospital has an Expected Number of CLABSIs of 0.5 for a one year period

• But they had 1 CLABSI

• SIR would calculate as 2.0

• SIR cannot be interpreted correctly when Expected Number is less than 1

• You’ll see “ ” in reports when Expected Number is less than 1

• Other data may be excluded from the SIR, depending on circumstance

34

Standardized Infection Ratio (SIR)

35

Standardized Infection Ratio (SIR) • Uses…

• CMS reporting for Inpatient Quality Reporting Program • Benchmark for Value-Based Purchasing • Benchmark for Hospital-Acquired Conditions Penalty Program

• National Action Plan to Prevent Health Care-Associated Infections: Roadmap to Elimination

• CDC National and State HAI Progress Reports

36

In-Plan SIRs vs. CMS-Reported SIRs

Beginning January 1, 2015

37

Statistical Significance

37

38

Statistical Significance

• “Statistical significance…” • Don’t be scared! • It’s just a measure of reliability.

• Attempts to answer these questions: • For SIRs

• Is the SIR really different from 1.0 ?

• For Rates • Is the rate really different from the pooled mean?

39

Statistical Significance p-Value • “p” is the probability that the difference or relationship occurred by

chance

• “Null hypothesis” (H0) is the assumption that there is no real difference between your value and the benchmark

• α (alpha) is the amount of error you’re willing to accept in rejecting the H0. NHSN uses α = 0.05 to reject H0

• Therefore, a p-Value < 0.05 is considered “significant”

• In other words, when p < 0.05, you can be 95% confident the difference between your statistic and the benchmark is reliable

• NHSN calculates the p-Value using a chi-square (χ2)-like test for SIRs and a proportions test for rates

40

Statistical Significance Confidence Intervals (CI) • A confidence interval (CI) provides us with a lower and upper limit of

values between which the SIR could fall if the entire population was measured.

• If 1.0 falls within the CI, the results are not statistically significant.

• 95% CIs are very sensitive to sample size; the smaller the size, the wider the interval

SIR 1.8

UL 2.1

LL 0.8

0.0

1.0

41

Statistical Significance

SIR 1.8

UL 2.9

LL 1.6

0.0

1.0

Confidence Intervals (CI) • A confidence interval (CI) provides us with a lower and upper limit of

values between which the SIR could fall if the entire population was measured.

• If 1.0 falls within the CI, the results are not statistically significant.

• 95% CIs are very sensitive to sample size; the smaller the size, the wider the interval

42

NHSN Statistics Calculator

43

Statistical Significance

• So what? • p-values and 95%-CIs are a measure of

reliability • Significance is rare, but when it is…pay

attention! • The significance means there is a true difference

between the two values • Has something changed? • Does something need your attention? • Have your improvement efforts paid off?

44

TAP Strategy and the CAD

45

CDC’s Innovation…

TAP Reports!

46

47

Goals…

http://www.hsdl.org/?view&did=739815

48

TAP Reports

• New statistic… “C A D” • Cumulative Attributable Difference

• May be a positive number (“excessive”) • May be a negative number (“prevented”)

49

TAP Reports

• Example #1 • 11 HO CDI • 9.354 Expected • HHS Target: 0.70

CAD = 11 – (9.354 * 0.70) CAD = 4.452 (positive)

• Example #2 • 3 HO CDI • 9.354 Expected • HHS Target: 0.70

CAD = 3 – (9.354 * 0.70) CAD = -3.548 (negative)

Note that the effect of the CAD is to lower the “Expected Number” by a given percentage to reach the HHS goal. (In this case, from 9.354 to 6.548)

50

TAP Reports

• Provide the facility-level CAD, by HAI type • Sum of all infections – (sum of all expected * HHS Target)

• Rank-order units (highest CAD first) within a facility

• Allows facilities to “Target” highest contributors to the overall facility CAD

51

numPathUTI: Number and pathogens isolated in the infections # (EC, YS, PA, KS, PM, ES)

EC = E. coli YS = yeast PA = P. aeruginosa KS = K. pneumoniae/oxytoca PM = Proteus mirabilis ES = Enterococcus species

52

53

With TAP Reports….

54

Q & A

55

For survey:

www.SurveyMonkey.com/s/VD3R9K9

For more information: www.Medicare.QualisHealth.org/projects/healthcare-

associated-infection

This material was prepared by Qualis Health, the Medicare Quality Innovation Network - Quality Improvement Organization (QIN-QIO) for Idaho and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C1-QH-1766-05-15

Contact Martha Jaworski, RN, MS, CIC

Idaho QI Consultant marthaj@qualishealth.org

208-383-5944

Jamie Moran, MSN, RN, CIC Washington QI Consultant jamiem@qualishealth.org

206-288-2512