1. Quarterly Reports for SHN’s Interventions Understanding and Using the Data to Guide Performance December 10, 2008
Virginia Flintoft, RN MSc
SHN! Central Measurement Team

2. Goal
To understand how to enter YOUR data into the Quarterly Reports (Monthly for ANC).
To know how to use the Quarterly/Monthly reports to interpret performance and guide your QI initiatives.

3. SHN Performance Reports
Data In …
Sending your results to
Central Measurement Team

4. Submitting Data from a CoP
Send you data from ANY COP
Click on “SHN Data Submission Home “ button to get to data submission screen

5. Submitting Data from a CoP
Save the url on this screen to your ‘favourites’ for easy access next month
1. Using ‘browse’ button select worksheet you want to send to CMT – 1 worksheet at a time.
2. Enter a message - include your name and phone or .
3. Click on ‘Upload/ Télèchargement’ button to send worksheet to CMT

6. SHN Performance Reports
Data Out…
Reports you get from
Central Measurement Team

7. SHN Performance Reports
Quarterly Reports
Apr-June, July-Sept., Oct.-Dec., Jan.-Mar
By measure
Distributed to Key Organizational Contact and Team Leaders
Allows team to monitor their performance against national rates and by implementation stage
Quarterly Performance
By individual team and measure
Distributed to Node leads and SIAs
Allows them to identify teams in need of support

8. Collaborative Performance Reports
Monthly Reports
Data in – 25th of each month
i.e. December data sent in January 25th
Data out – one week later
Allows team to monitor their performance against Collaborative average rates

9. Next ANC-Report Oct. Data Submission Cut-off Dec.5 ANC Report
Distributed

10. ANC Reports ANC-Report format ANC – Report # Date data submitted
Months reported* 1
Nov’08
June ’08 – Oct.’08 2
Dec.’08
June ’08 – Nov.’08 3
Jan.’09
June ’08 – Dec.’08 4
Feb.’09
June ’08 – Jan.’09
* Cumulative report

11. Collaborative Performance Reports
Monthly Reports for ANC
Rate of Undocumented Intentional Discrepancies
Rate of Unintentional Discrepancies
Percent Residents Reconciled on Admission

12. SHN Monthly/Quarterly Reports
What to make of your results -
Interpreting the
Monthly/Quarterly Reports

13. ANC Collaborative Reports
The Basics
Data submission cut off for ANC-Reports
25th of each month
ANC-Reports issued
Week following data submission cut off
Sent to Dannie and Theresa for distribution to teams

14. Name and definition of measure
Date of Data – months of data covered on this report
Date Created – actual date when analysis program ran
3 Worksheets within each Workbook –
QR spreadsheet;
Chart – ANC vs Team Result;
Chart - Nodes vs Team (QR only)

15. Warning - Small Sample Size... 5 or less – results unreliable
Examples of Small Sample Size...

16. QR Data Analyzed & Presented: Nationally by Node – Atlantic Ontario
ANC
National will be called “Atlantic
Collaborative” and data presented
will be for ANC members ONLY
Nodes will be deleted
Implementation Stage as reported
QR Data Analyzed & Presented:
Nationally
by Node –
Atlantic
Ontario
Quebec
Western
Paeds
by Implementation Stage –
Baseline
Early (Working to Goal)
Full

17. Enter Goal for term of collaboration from your worksheet
Entering “Local Team” Data:
Copy values in “Final Calculation” row on Data
Entry Sheet of Measurement Worksheet and
Paste into “Local Team” row (r7) on Quarterly Report

19. Definitions:
“N (Teams)” = Number of teams reporting data for a specific month
“Mean” = average score for the measure for the hospitals reporting data for that month.
“SD” = standard deviation - a value indicating how widely dispersed the scores are around the mean; a measure of variation.
Minimum – lowest score reported for a specific month
Maximum - highest score reported for a specific month
25th – 75th percentile – of all teams reporting data for that month X% have a lower score than the specific percentile rank.

20. Quarterly Report format
QR #1 – data submitted for Nov ‘05 - June ‘06
QR #2 – data submitted For Nov ‘05 - Sept. ‘06
QR #7 – data submitted for Nov ‘05 - Dec. ‘07
Data becomes more stable as sample size increases

21. Interpretation:
Lower and Upper Bound = 95% CI
Compare your performance (Local Team) to National mean month-over-month
Compare your performance to CI
if score falls between upper and lower bound = statistically similar (Jan.’08 – Jun.’08)

22. Interpretation:
Lower and Upper Bound = 95% CI
Compare your performance (Local Team) to National mean month-over-month
Compare your performance to CI
if score falls between upper and lower bound = statistically similar (Jan.’08 – Jun.’08)

23. Making Sense of Quarterly Reports
Quarterly Reports - Run Charts

24. Above ANC average (poor performance – Med Rec)
Same as ANC average (average performance)
Below ANC average (good performance – Med Rec)

26. Quarterly Reports
Interpreting and Reporting Your Results

27. Making Sense of Quarterly Reports
Sharing the Quarterly Reports
With who do you (KOCs + Team Leaders) share the QR info?
Team
Staff
Director / Senior Management
(Board)

28. SHN Performance Report – Example 1
Poor
Good
Great
Medication Reconciliation
Ventilator-Associated Pneumonia
Measure: 1.0 Mean Number of Undocumented Intentional Discrepancies
Current Qtr = 0.64
Goal = 0.28
# Mos. Reporting = 21
Node Qtr = 0.58
Nat. Qtr = 0.54
Measure: 2.0 Mean Number of Unintentional Discrepancies
Current Qtr = 0.09
Goal = 0.09
# Mos. Reporting = 21
Node Qtr = 0.68
Nat. Qtr = 0.63
Measure: 1.0 VAP Rate in ICU per 1000 Ventilator Days
Measure: 2.0 VAP Bundle Compliance
Current Qtr = 0.64
Goal = 0.28
# Mos. Reporting = 21
Node Qtr = 0.58
Nat. Qtr = 0.54
Current Qtr = 78%
Goal = 95%
# Mos. Reporting = 10
Node Qtr = 85%
Nat. Qtr = 83%
Element: Head of Bed
Qtr.= 97%
Element: Sedation Vacation
Qtr.= 60%
Measure: 3.0 Medication Reconciliation Success Index
Measure: 4.0 Medication Reconciliation at Discharge
Current Qtr = 92%
Goal = 90%
# Mos. Reporting = 21
Node Qtr = 89%
Nat. Qtr = 93%
No Data
Element: Oral v Nasal
Qtr.= 77%
Element: EVAC Tube
No EVAC Tubes

29. SHN Performance Report – Example 2

30. How to Get Answers to your SHN Measurement Questions
Central Measurement Team
Safety Improvement Advisors (SIAs)
Community of Practice
Faculty members
Clinical advisors
Improvement advisors
Conference calls on interventions

31. How to Get Answers to your Measurement Questions
Safety Improvement Advisors (SIAs)
Community of Practice
Central Measurement Team
Faculty members
Clinical advisors
Improvement advisors
Collaborative Conference calls

32. Questions or More Information
Virginia Flintoft
G. Ross Baker Alexandru Titeu