1. Scorecards & Visual Display of Data
11/14/2018 6:03 AM
Scorecards & Visual Display of Data

2. Why develop a scorecard?
One of the key roles of a Board and Senior Leadership team is to oversee an organization’s performance against standards set either externally, internally or both. A Scorecard that is reviewed at least quarterly is a good vehicle for supporting this critical function.
If an organization has a Quality Committee, there may be a desire to create a separate Quality Scorecard; alternatively, quality indicators can be integrated into an organization’s existing scorecard.
Once the Quality Plan (QIP) is developed, the organization’s Scorecard(s) are the vehicle for monitoring progress on achieving the targets established in the Quality Plan.

3. Customer Satisfaction
Relationship between a “Corporate” Scorecard and a Quality Scorecard: Comparison of Dimensions & Indicators
Client Experience with Care Coordination
Strategic Plan
Serious Safety Events
Customer Satisfaction
Internal Process
Finance
Learning & Growth (HR, Partnerships)
Falls, Infections, Medication Errors, all Resulting in Harm
Person-centred
Effective
Safe
Quality Plan
Overall Client Experience
Integrated
Access

4. Sample Indicators by Dimension, for Consideration
Safe
Effective
Accessible
Client Experience
Integrated
# Adverse Events (incidents resulting in harm)
Falls
Pressure Ulcers
Medication Errors
Link to ROPs for CSS
[# Staff Injuries Resulting in Absenteeism]
% Unplanned Hospitalizations for Clients on Service
% Clients with High/Very High MAPLe Scores Supported by CSS in the Community
% Clients with Stable or Improved Functional Ability
% Clients with All Goals Met at Discharge
# Days from Referral to Assessment
# Days from Referral to Treatment
Average # Days on Wait List
# People Waiting for Service
Time from Referral to CSS First Visit
Service Denial
% Clients Satisfied with Services Received
Questions related to:
Provider Continuity
Engagement of Client/Family in Goal Setting
Overall Satisfaction
Satisfaction with Transition Planning
% Unplanned Hospitalizations for Clients on Service
Readmissions to Hospital for CSS Clients within 30 Days of Discharge
Repeat ED Visits for Clients Classified as CTAS 4, 5

5. Into which Quality Dimension would you include the following indicators?
1. Safe
2. Effective
3. Accessible
4. Client Experience
5. Integrated
# Days from Referral to Service Initiation
Client Experience with Continuity of Service Personnel
# Complaints
Service Denials
% Clients Who Improve or Maintain their Level of Functional
% Clients With Repeat Visits to the Emergency Department
% Clients Whose Level of Functional Independence is Maintained or Improved

6. Guidelines for Developing a Scorecard
Rule of thumb….ideally no more than 20-25….(Niven, 2002)
Include indicators where there is a gap between current and target
Focus on outcomes for Scorecard
Ensure the indicators are Actionable
Balance across dimensions (don’t want to optimize in one area to the detriment of other areas)
Align with:
Strategy
High level quality aims
Provincial/LHIN priorities
Other key external accountabilities
Long Term Goal:
Keep these to a minimum;
Include if outside of
acceptable/desired range,
But otherwise ensure
someone is reviewing…include
only if outside range

7. Strategy for defining metrics for a streamlined, outcomes-based quality scorecard….
Measures Related to High Level Quality Aims
MSAA, HQO, Accreditation, etc
Filter:
Are you outside an acceptable corridor?
Is this metric so important, it must always be on the Board’s radar?
Quality Scorecard
Access
Effective
Integration
Safe
Patient-
Centred

8. Ideal format for an organizational scorecard
Front page – Colour-coded dashboard
Line graphs/Run Charts on subsequent pages
Monthly data points even if the Scorecard is only reviewed quarterly

9. Rule One: Shift
Six or more consecutive points either all above or all below the median
Values that fall on the median neither add to nor break a shift…skip them and continue counting

10. Two of the most common rules for interpreting Run Charts
Rule One: Shift
Client Throughput
Days 25 20 15 10 5
Signal of Improvement
Shift: 6 or more points above median

11. Rule Two: Trend
Five or more consecutive points all going up or all going down
If the value of two or more consecutive points is the same, ignore one of the points and continue counting
Note: the starting point doesn’t count
Note: either there is a trend (5 points) or there is not – there is no such thing as “trending”

12. Rule Two: Trend Signal of Improvement
Trend: 5 or more points ascending (may or may not cross median)
Client Throughput
Days 25 20 15 10 5
Client Throughput

13. On the following graph, is there a signal of significant change (are one of the rules met) and are the data moving in the right direction?
1. Is there:
A Shift
A Trend
Neither
Not sure
2. Are the data moving in the right direction?
Yes No
Staying the same

14. Number of Clients Waiting for Service
On the following graph, is there a signal of significant change (are one of the rules met) and are the data moving in the right direction?
Number of Clients Waiting for Service 80 70 60 50 40 30
1. Is there:
A Shift
A Trend
Neither
Not sure
2. Are the data moving in the right direction?
Yes No
Staying the same

15. On the following graph, is there a signal of significant change (are one of the rules met) and are the data moving in the right direction?
Percent of Clients Receiving First Home Visit Within 3 Days of Referral
1. Is there:
A Shift
A Trend
Neither
Not sure
2. Are the data moving in the right direction?
Yes No
Staying the same

16. To summarize….
The quality dimensions (from ECFAA) and that can be used in a Quality Scorecard are aligned with the Internal Process and Customer Satisfaction quadrants of a traditional Corporate Scorecard.
There are many outcome indicators that align with each of the five ECFAA dimensions of quality. Organizations should include no more than indicators in a scorecard, balanced across the dimensions.
Line graphs (Run Charts or Statistical Process Control Charts) are the best method for displaying data for improvement.