developing a valid surveillance design (or…is there life after mandatory reporting?) ona...
TRANSCRIPT
Developing a Valid Surveillance Design
(Or…Is there life after mandatory reporting?)
Developing a Valid Surveillance Design
(Or…Is there life after mandatory reporting?)
Ona MontgomeryOna Montgomery
Amarillo VA Health Care SystemAmarillo VA Health Care System
Proposed Initial Mandatory Reporting ComponentsProposed Initial Mandatory Reporting Components CL-BSICL-BSI
NICU - birthweight stratifiedNICU - birthweight stratified ICU – stratified by type of ICUICU – stratified by type of ICU Special Care Units – stratified by type of unitSpecial Care Units – stratified by type of unit
SSI SSI CMS procedures CMS procedures Stratified by NHSN indexStratified by NHSN index ASC and general hospitalsASC and general hospitals
HCA-RSV HCA-RSV Pedi hospitalsPedi hospitals General hospitals with pedi unitsGeneral hospitals with pedi units
Other Surveillance DriversOther Surveillance Drivers
Regulations and requirements Regulations and requirements JCAHOJCAHO CMSCMS Corporate mandatesCorporate mandates
Internal strategic plansInternal strategic plans
Single acute care facility or part of larger Single acute care facility or part of larger system?system?
How do we develop a scientifically valid and rational surveillance design when so many aspects seem out of our control?
(As well as so much competition for available personnel and other finite resources)
Approach with an epidemiologic perspective.
Recommended Practices for Surveillance
1.1. Assess the populationAssess the population
2.2. Select the outcome or process to surveySelect the outcome or process to survey
3.3. Apply surveillance definitionsApply surveillance definitions
4.4. Collect surveillance dataCollect surveillance data
5.5. Calculate rates & analyze surveillance Calculate rates & analyze surveillance findingsfindings
6.6. Apply risk stratification methodsApply risk stratification methods
7.7. Report and use surveillance informationReport and use surveillance information
APIC Surveillance Initiative Working Group, 1998 (Under revision)
Key Epidemiologic Principle of SurveillanceKey Epidemiologic Principle of SurveillanceFocus on Focus on
populations or populations or individuals at risk individuals at risk for infection.for infection.
Organizational population assessment should be comprehensiveOrganizational population assessment should be comprehensive
Healthcare VenuesHealthcare Venues
Acute careAcute care ICUsICUs LTCLTC RehabRehab LTACLTAC AmbulatoryAmbulatory Home careHome care
Patient PopulationsPatient Populations ElderlyElderly Trauma Trauma Diabetic Diabetic Oncology Oncology DialysisDialysis Spinal cord injurySpinal cord injury Other Important risk Other Important risk
populationspopulations
Epidemiologic PrincipleEpidemiologic Principle
Focus on priority problems/improvement Focus on priority problems/improvement objectivesobjectives Greatest potential for improvement (high Greatest potential for improvement (high
volume, high risk, high intervention impact) volume, high risk, high intervention impact) Significant outcomeSignificant outcome Organizational objectives Organizational objectives Cost-effectivenessCost-effectiveness Risk managementRisk management Existence of standards and mandatesExistence of standards and mandates
Surveillance Practice 1:
Assess the PopulationSurveillance Practice 1:
Assess the Population
Obtain data to describe / understand Obtain data to describe / understand YOUR patient population YOUR patient population
Establish priorities for surveillance Establish priorities for surveillance through your annual risk assessmentthrough your annual risk assessment
Which patients or staff in your Which patients or staff in your organization are at increased risk for organization are at increased risk for infection? infection?
Surveillance Practice 2:
Select Outcome or Process to Survey
Surveillance Practice 2:
Select Outcome or Process to SurveyConsider...Consider...• Relative frequency of the event (high volume)Relative frequency of the event (high volume)
• Cost or impact of a specific negative outcome (high Cost or impact of a specific negative outcome (high risk)risk)
• Preventability (Preventability (high intervention impact)high intervention impact)
• Cost-benefit Cost-benefit
• Customer needs (e.g. priorities set by facility plan, Customer needs (e.g. priorities set by facility plan, regulatory compliance)regulatory compliance)
• Organizational mission / strategic goalsOrganizational mission / strategic goals
• Available resources (administrative commitment) Available resources (administrative commitment) Cost-effectivenessCost-effectiveness
The Surveillance ProcessThe Surveillance Process
Assessing populations for riskAssessing populations for risk Choosing problem or event to be studiedChoosing problem or event to be studied Often done simultaneouslyOften done simultaneously
Factor in mandatory reporting componentsFactor in mandatory reporting components
Outcome and Process MeasuresOutcome and Process Measures
OutcomesOutcomes InfectionsInfections EventsEvents
ProcessesProcesses Known to be related to outcomeKnown to be related to outcome Key patient care stepsKey patient care steps BundlesBundles
Search of literature, opinions of experts, or Search of literature, opinions of experts, or practice standards may point to appropriate practice standards may point to appropriate process indicatorsprocess indicators
Process Measures - ExamplesProcess Measures - Examples Prophylactic antibiotic timing in ambulatory surgeryProphylactic antibiotic timing in ambulatory surgery Antibiotic selection in treatment of outpatient Antibiotic selection in treatment of outpatient
pneumoniapneumonia Immunization rates in all settings - staff and patientsImmunization rates in all settings - staff and patients Adherence to disinfection processes in dialysis centerAdherence to disinfection processes in dialysis center Compliance with bundles prevent ventilator pneumoniaCompliance with bundles prevent ventilator pneumonia Device utilization ratios - with or without device-Device utilization ratios - with or without device-
associated infection rates in home care, LTC, ICUsassociated infection rates in home care, LTC, ICUs PPD skin testing compliance of patients and/or staff - in PPD skin testing compliance of patients and/or staff - in
conjunction with outcome measure of PPD skin test conjunction with outcome measure of PPD skin test conversionsconversions
Surveillance Practice 4:
Collect Surveillance DataSurveillance Practice 4:
Collect Surveillance Data Data collection methods must be Data collection methods must be
systematicsystematic
Utilize available information systems Utilize available information systems (including electronic data)(including electronic data)
The team approach is critical - The team approach is critical - negotiate for assistancenegotiate for assistance
Provide surveillance personnel with Provide surveillance personnel with standardized trainingstandardized training
Intensity of surveillance must be Intensity of surveillance must be maintained over time!maintained over time!
Surveillance Practice 5:
Calculate Rates and Analyze Findings
Surveillance Practice 5:
Calculate Rates and Analyze Findings Plan for data analysis. Plan for data analysis.
Express surveillance information in numerical Express surveillance information in numerical terms (I.e. incidence density, rates) terms (I.e. incidence density, rates)
Turn surveillance “data” into “information”Turn surveillance “data” into “information”
Use statistical probability methods to determine Use statistical probability methods to determine whether observed differences in rates are whether observed differences in rates are meaningful (i.e. significance testing; p values)meaningful (i.e. significance testing; p values)
Provide analysis and interpretation Provide analysis and interpretation
Use scientifically sound, well understood methods in data collection and analysis
Use scientifically sound, well understood methods in data collection and analysis
Calculate rates only when numbers are Calculate rates only when numbers are sufficientsufficient
Numerator analysis and reporting may be Numerator analysis and reporting may be appropriate for infrequent events appropriate for infrequent events
Control chart theory has been applied to Control chart theory has been applied to rate- and event-based datarate- and event-based data
0
50
100
150
200
250
300
350
03/0
1/01
06/0
6/01
08/2
1/01
12/2
0/01
05/0
6/02
07/1
8/02
08/2
3/02
10/2
1/02
11/0
5/02
02/1
9/03
04/0
2/03
06/1
0/03
09/0
4/03
04/2
1/04
09/2
7/04
11/1
7/04
12/0
2/04
12/0
8/04
01/1
1/05
03/0
9/05
08/0
9/05
08/1
6/05
08/2
4/05
09/1
4/05
01/0
4/06
01/1
1/06
02/2
2/06
Date of Surgery
# Ca
ses
betw
een
Mean = 86
Post-op PneumoniaTrend Analysis
mean calc based on FY01 - FY04
GOOD
Surveillance Practice 6:
Apply Risk Stratification Methods
Surveillance Practice 6:
Apply Risk Stratification MethodsRisk stratification: subdividing (stratifying) Risk stratification: subdividing (stratifying)
your surveillance population into groups at your surveillance population into groups at similar levels of infection risk prior to similar levels of infection risk prior to performing any analyses or comparisons.performing any analyses or comparisons.
To ensure comparing “apples to apples”
CDC NNIS Risk Index for SSI Surveillance
CDC NNIS Risk Index for SSI Surveillance
Patient-specific Risk ScorePatient-specific Risk Score Total 0-3 points Total 0-3 points
Wound class class III or IVWound class class III or IV 1 point1 point
ASA score 3, 4, 5ASA score 3, 4, 5 1 point1 point
Duration of surgery > cutpointDuration of surgery > cutpoint 1 point 1 point
CDC NNIS Risk Stratification for High Risk Nursery (HRN)
Surveillance
CDC NNIS Risk Stratification for High Risk Nursery (HRN)
Surveillance
Stratification by Birthweight Categories:Stratification by Birthweight Categories:
• </= 500 grams</= 500 grams
• 501 - 1000 grams501 - 1000 grams
• 1001-1500 grams1001-1500 grams
• 1501-2500 grams1501-2500 grams
• >2500 grams>2500 grams
Surveillance Practice 7:
Report and Use Surveillance Data
Surveillance Practice 7:
Report and Use Surveillance Data
Plans for the distribution of Plans for the distribution of surveillance information should be surveillance information should be incorporated into the development incorporated into the development of each surveillance objectiveof each surveillance objective
Provide feedback to Provide feedback to cliniciansclinicians!!
Reporting should be timelyReporting should be timely
Present surveillance findings using Present surveillance findings using graphs and easy-to-read tablesgraphs and easy-to-read tables
Present surveillance data in a manner to Present surveillance data in a manner to stimulate ideas for process improvement.stimulate ideas for process improvement.
Perform follow-up surveillance to monitor for Perform follow-up surveillance to monitor for improvement following changes (“close the improvement following changes (“close the loop”).loop”).
“Surveillance without action should be abandoned.”
Compared to what?Compared to what?
Internal changes over timeInternal changes over time External comparisons such as:External comparisons such as:
NNIS/NHSNNNIS/NHSN IHI publicationsIHI publications State reporting dataState reporting data
Pooled means and percentiles of the distribution of device-associated infection rates, by type of
ICU, NNIS, 1/95-6/03
Central line-associated BSI rate** PercentileNo. of Central Line- Pooled 10% 25% 50% 75% 90%
Type of ICU Units Days Mean (median)
Coronary 114 363,976 4.2 0.0 1.9 4.2 5.8 8.4
Cardiothoracic 71 598,118 2.9 0.4 1.3 2.2 3.5 4.9
Medical 143 975,318 5.7 2.1 3.4 5.0 6.8 9.6
Medical-Surgical
Major teaching 133 936,223 5.0 2.2 3.0 4.9 6.3 7.7
All others 187 1,295,477 3.7 0.0 1.8 3.3 5.0 6.8
Neurosurgical 52 180,581 4.8 0.0 2.5 4.1 6.5 9.0
Pediatric 79 428,104 7.3 0.7 3.8 5.9 8.8 11.5
Surgical 160 1,267,959 5.2 1.1 2.6 4.7 6.9 9.3
Trauma 28 178,179 7.8 2.5 5.2 6.6 10.0 12.3** Number of central line-associated BSI Number of central line-days
X 1000
Pooled means and percentiles of the distribution of device utilization ratios, by type of ICU,
NNIS, 1/95-6/03
Central line utilization** PercentileNo. of Pooled 10% 25% 50% 75% 90%
Type of ICU Units Patient-Days Mean (median)
Coronary 115 1,120,967 0.32 0.13 0.21 0.29 0.42 0.58
Cardiothoracic 71 751,547 0.80 0.57 0.70 0.82 0.91 0.95
Medical 143 1,905,674 0.51 0.30 0.37 0.52 0.64 0.75
Medical-Surgical
Major teaching 133 1,688,840 0.55 0.35 0.45 0.55 0.64 0.73
All others 187 2,770,191 0.47 0.25 0.34 0.47 0.57 0.63
Neurosurgical 52 401,236 0.45 0.26 0.38 0.49 0.55 0.63
Pediatric 82 936,169 0.46 0.20 0.30 0.41 0.53 0.60
Surgical 160 1,958,691 0.65 0.44 0.55 0.67 0.76 0.86
Trauma 28 280,074 0.64 0.47 0.57 0.65 0.75 0.85** Number of central line-days Number of patient-days
Surgical Prophylaxis (% given < 1 hr. before incision)Surgical Prophylaxis (% given < 1 hr. before incision)
0102030405060708090
100
1QFY02
2QFY02
3QFY02
4QFY02
3-Jan 3-Feb 3-Mar
% compliance
Development of your Hospital Surveillance Plan
Development of your Hospital Surveillance Plan
Plan must be written!Plan must be written! Reflect logical risk assessmentReflect logical risk assessment Address internal and external prioritiesAddress internal and external priorities Describe each surveillance componentDescribe each surveillance component
• PopulationsPopulations• Outcomes of concernOutcomes of concern• Rationale (e.g., risk assessment rating)Rationale (e.g., risk assessment rating)• Measurement/analysisMeasurement/analysis• Interventions and reporting planInterventions and reporting plan
Risk Rating ExampleRisk Rating Example
Probability Risk Current Systems
4 Expect it to happen5 Loss of life/
limb/function 5 None in place currently
3 Likely4 Temp loss of
function 4 Poor
2 Maybe3 Prolonged
hospitalization 3 Fair
1 Rare2 Moderate clinical /
financial 2 Good
0 Never1 Minimal clinical /
financial1 Solid system in place
currently
Plan ExamplePlan ExamplePriorit
yRisk
Score
Surveillance focus
(Populations affected)
Outcome Goal Intervention Plans Evaluation Methods Progress
1 38 Influenza Prevention and Pandemic Preparedness
(Populations: Acute, long term, & ambulatory care; HCWs)
Influenza prevention in all populations
SEASONAL INFLUENZA1. Pre-season CNE2. Revise NH on-line module3. Staff immunization
strategies4. Prevention/control of NH
outbreaksPANDEMIC PREPAREDNESS- Tests of systems- Supply stockpile
Vaccination acceptance in long term care, ambulatory (including COBCs), inpatients, and staff
Preparedness for local epidemic and/or pandemic (surge capacity)
Detection, prevention and control of nursing home outbreaks
Development of internal pandemic response plan linked to community plan
2 25.5 CL-BSI Risk Reduction
(Populations: Acute, long term & ambulatory care)
Prevention of Central Line associated Blood-stream Infection
1. Expand bundle compliance monitoring to entire facility
2. Revise policies & procedures3. Improve maintenance
process (e.g. pt. education, dressings, showers, securement)
4. Implement CHG dressing change kits
Outcome: Central Line associated Bloodstream Infection in ICU
Process: Compliance with the “bundle” of prevention measures
Outpatient processes and outcomes
Explore feasibility of non-ICU line day collection
3 25 Dialysis (if initiated in
FY07)(Population: Acute
care; HCW)
Prevention of transmission of BBP and other dialysis-related infection
1. Monitoring c/w CDC NHSN system
2. Facilitate IC involvement in P&P development
3. Implement H2O monitoring
Outcome: BSI in Dialysis patients
Process: Compliance with defined procedures
Key PrincipleKey Principle
Include clinicians in all steps of Include clinicians in all steps of planningplanning and implementing surveillance plan.and implementing surveillance plan. Improved buy-in and participationImproved buy-in and participation Non-acute settings -- technical staffNon-acute settings -- technical staff
ExamplesExamples ICU - front line staff (bundles)ICU - front line staff (bundles) LTC – CNAs as well as nursesLTC – CNAs as well as nurses Home health - family educatorsHome health - family educators
Key PrincipleKey Principle
Report data to clinicians who can influence Report data to clinicians who can influence patient care practices.patient care practices. Important tie between surveillance and clinical Important tie between surveillance and clinical
carecare Reinforces staff involvementReinforces staff involvement Allows completion of surveillance cycleAllows completion of surveillance cycle
Fostering Effective InterventionsFostering Effective Interventions
Use stakeholders and content expertsUse stakeholders and content experts Effective? Effective? Feasible?Feasible? Barriers?Barriers?
ICPs should strive to facilitate, not dictate ICPs should strive to facilitate, not dictate or implementor implement
Document improvement activities Document improvement activities undertaken by all participantsundertaken by all participants
Follow-upFollow-up
Continued surveillance for internal Continued surveillance for internal comparisons to measure improvementcomparisons to measure improvement
Outcome monitoring - look for improved Outcome monitoring - look for improved infection ratesinfection rates
Process monitoring - look for improved Process monitoring - look for improved rates of compliance - important even rates of compliance - important even without corresponding change in outcome without corresponding change in outcome ratesrates
“A good surveillance system does not guarantee you will make the right decisions, but it reduces the chances of making the wrong ones.”
Dr. Alexander Langmuir
N Eng J Med 1963;268:182-92.
References and ResourcesReferences and Resources Guidance on Public Reporting of Healthcare-Associated Infections
AJIC Feb. 2005 http://www.cdc.gov/ncidod/dhqp/pdf/hicpac/
PublicReportingGuide.pdf Lee TB, Baker OG, Lee JT, Scheckler WE, et al. Recommended
Practice for Surveillance. American Journal of Infection Control 1998;26:277-288.http://www.apic.org/AM/Template.cfm?
Section=Surveillance_Definitions_Reports_and_Recommendations&Template=/CM/ContentDisplay.cfm&ContentFileID=2710
NHSN Member site (surveillance protocols, etc.):http://www.cdc.gov/ncidod/dhqp/nhsn_members.html
Gaynes R, Richards C, Edwards J, et al. Feeding Back Surveillance Data to Prevent Hospital-acquired Infections. Emerging Infectious Diseases Mar-Apr 2001http://www.cdc.gov/ncidod/eid/vol7no2/pdfs/gaynes.pdf
Haley RW. Surveillance by objective: a new priority-directed approach to the control of nosocomial infections. Am J Infect Control 1985;13:78-89.