quarterly reports for shn’s interventions understanding and using the data to guide performance...
TRANSCRIPT
Quarterly Reports for SHN’s Interventions
Understanding and Using the Data to Guide Performance
December 10, 2008Virginia Flintoft, RN MSc
SHN! Central Measurement Team
Goal• To understand how to enter YOUR data
into the Quarterly Reports (Monthly for ANC).
• To know how to use the Quarterly/Monthly reports to interpret performance and guide your QI initiatives.
SHN Performance Reports
Data In …
Sending your results to
Central Measurement Team
Submitting Data from a CoP Send you data from
ANY COP
Click on “SHN Data Submission Home “
button to get to data submission
screen
Save the url on this screen to your ‘favourites’ for easy access next month
Submitting Data from a CoP
1. Using ‘browse’ button select worksheet you want to send to CMT – 1 worksheet at a time.
2. Enter a message - include your name and phone or email.
3. Click on ‘Upload/ Télèchargement’ button to send worksheet to CMT
SHN Performance Reports
Data Out…
Reports you get from
Central Measurement Team
SHN Performance Reports• Quarterly Reports
• Apr-June, July-Sept., Oct.-Dec., Jan.-Mar
• By measure
• Distributed to Key Organizational Contact and Team Leaders
• Allows team to monitor their performance against national rates and by implementation stage
• Quarterly Performance • By individual team and measure
• Distributed to Node leads and SIAs
• Allows them to identify teams in need of support
Collaborative Performance Reports
Monthly Reports• Data in – 25th of each month
• i.e. December data sent in January 25th
• Data out – one week later
• Allows team to monitor their performance against Collaborative average rates
Oct. Data Submission
Cut-off
ANC ReportDistributed
Next ANC-Report
Dec.5
ANC ReportsANC-Report format
ANC – Report
#
Date data submitted
Months reported*
1 Nov’08 June ’08 – Oct.’08
2 Dec.’08 June ’08 – Nov.’08
3 Jan.’09 June ’08 – Dec.’08
4 Feb.’09 June ’08 – Jan.’09
* Cumulative report
Collaborative Performance Reports
Monthly Reports for ANC• Rate of Undocumented Intentional Discrepancies
• Rate of Unintentional Discrepancies
• Percent Residents Reconciled on Admission
SHN Monthly/Quarterly Reports
What to make of your results -
Interpreting the
Monthly/Quarterly Reports
The Basics
• Data submission cut off for ANC-Reports• 25th of each month
• ANC-Reports issued • Week following data submission cut off
• Sent to Dannie and Theresa for distribution to teams
ANC Collaborative Reports
Name and definition of
measure
Date of Data – months of data covered on this
report
Date Created – actual date when analysis
program ran
3 Worksheets within each Workbook –i.QR spreadsheet; ii.Chart – ANC vs Team Result; iii.Chart - Nodes vs Team (QR only)
Warning - Small Sample Size... 5 or less – results unreliable
Examples of Small Sample Size...
QR Data Analyzed & Presented:• Nationally• by Node –
Atlantic Ontario Quebec Western Paeds
• by Implementation Stage – Baseline Early (Working to Goal) Full
ANC• National will be called “Atlantic Collaborative” and data presented will be for ANC members ONLY• Nodes will be deleted• Implementation Stage as reported
Enter Goal for term of collaboration from your worksheet
Entering “Local Team” Data:• Copy values in “Final Calculation” row on Data Entry Sheet of Measurement Worksheet and • Paste into “Local Team” row (r7) on Quarterly Report
Definitions:
• “N (Teams)” = Number of teams reporting data for a specific month
• “Mean” = average score for the measure for the hospitals reporting data for that month.
• “SD” = standard deviation - a value indicating how widely dispersed the scores are around the mean; a measure of variation.
• Minimum – lowest score reported for a specific month
• Maximum - highest score reported for a specific month
• 25th – 75th percentile – of all teams reporting data for that month X% have a lower score than the specific percentile rank.
Comparator Group Descriptive Statistic Nov'05 Dec'05 Jan'06 Feb"06 Mar'06 Apr'06 May'06Score Score Score Score Score Score Score
Aspirin at Arrival Goal >90% >90% >90% >90% >90% >90% >90%Local Team (input own data) Final Calculation 75% 82% 83% 79% 88% 90% 90%
National N (Teams) 17 11 6 6 10 3 4Mean 94.06% 98.10% 98.15% 97.22% 98.89% 77.77% 95.50%
SD 6.66% 4.09% 4.53% 6.82% 3.51% 38.51% 5.47%
Quarterly Report format
Comparator Group Descriptive Statistic Nov'05 Dec'05 Jan'06 Feb"06 Mar'06 Apr'06 May'06 Jun'06Score Score Score Score Score Score Score Score
Aspirin at Arrival Goal >90% >90% >90% >90% >90% >90% >90% >90%Local Team (input own data) Final Calculation
National N (Teams) 27 19 15 17 18 11 11 10Mean 92.63% 93.91% 89.07% 92.09% 98.96% 93.94% 98.36% 97.90%
SD 8.39% 11.82% 27.03% 24.22% 3.10% 20.11% 3.76% 4.99%
QR #1 – data submitted for Nov ‘05 - June ‘06
QR #2 – data submitted For Nov ‘05 - Sept. ‘06
• Data becomes more stable as sample size increases
QR #7 – data submitted for Nov ‘05 - Dec. ‘07
Interpretation:
• Lower and Upper Bound = 95% CI
• Compare your performance (Local Team) to National mean month-over-month
• Compare your performance to CIo if score falls between upper and lower bound = statistically similar
(Jan.’08 – Jun.’08)
Interpretation:
• Lower and Upper Bound = 95% CI
• Compare your performance (Local Team) to National mean month-over-month
• Compare your performance to CIo if score falls between upper and lower bound = statistically similar
(Jan.’08 – Jun.’08)
Making Sense of Quarterly Reports
Quarterly Reports - Run Charts
Above ANC average (poor performance – Med Rec)
Same as ANC average (average performance)
Below ANC average (good performance – Med Rec)
Quarterly Reports
Interpreting and Reporting Your Results
Sharing the Quarterly Reports• With who do you (KOCs + Team Leaders)
share the QR info?• Team
• Staff
• Director / Senior Management
• (Board)
Making Sense of Quarterly Reports
SHN Performance Report – Example 1
Measure: 3.0 Medication Reconciliation Success
Index
Measure: 4.0 Medication Reconciliation at
Discharge
Poor GreatGood
Measure: 1.0 Mean Number of Undocumented
Intentional Discrepancies
Current Qtr = 0.64 Goal = 0.28# Mos. Reporting = 21
Node Qtr = 0.58Nat. Qtr = 0.54
Medication Reconciliation
No DataCurrent Qtr = 92%Goal = 90%# Mos. Reporting = 21
Node Qtr = 89%Nat. Qtr = 93%
Ventilator-Associated Pneumonia
Measure: 1.0 VAP Rate in ICU per 1000 Ventilator
Days
Measure: 2.0 VAP Bundle Compliance
Current Qtr = 0.64 Goal = 0.28# Mos. Reporting = 21
Node Qtr = 0.58Nat. Qtr = 0.54
Current Qtr = 78%Goal = 95%# Mos. Reporting = 10
Node Qtr = 85%Nat. Qtr = 83%
Element: Head of Bed
Qtr.= 97%
Element: Sedation Vacation
Qtr.= 60%
Element: Oral v Nasal
Qtr.= 77%
Element: EVAC TubeNo EVAC
Tubes
SHN Performance Report – Example 2
How to Get Answers to your SHN Measurement Questions
• Central Measurement Team
• Safety Improvement Advisors (SIAs)
• Community of Practice
• Faculty members• Clinical advisors
• Improvement advisors
• Conference calls on interventions
How to Get Answers to your Measurement Questions
• Safety Improvement Advisors (SIAs)
• Community of Practice
• Central Measurement Team
• Faculty members• Clinical advisors
• Improvement advisors
• Collaborative Conference calls
Questions or More Information
Virginia Flintoft
416.946.8350
G. Ross Baker Alexandru Titeu
416.978.7804 416.946.3103