1. Monitoring and Reporting Performance Metrics
Improvement Path Systems – Joe Zuchora

2. Context, Purpose, Outcome
1) Context: Navy Medicine can benefit from monitoring and reporting performance metrics to help drive better decisions at all management levels.
2) Purpose: Guidance will be provided in designing and creating dashboards, and examples of successful dashboards used for performance monitoring will be discussed
3) Outcome: Identification of the need for dashboards and methodology for creating and using dashboards.
FOR OFFICIAL USE ONLY

3. Agenda What is a Dashboard? Why Dashboards? Monitoring of performance
Dashboard design
Metric selection and creation
Display of metrics
Setting targets
Management decisions
Case study
FOR OFFICIAL USE ONLY

4. What is a Dashboard? -Peter McFadden, CEO Excel Dashboard Widgets
“An easy to read, often single page, real-time user interface, showing a graphical presentation of the current status and historical trends of an organization’s Key Performance Indicators”
-Peter McFadden, CEO Excel Dashboard Widgets
FOR OFFICIAL USE ONLY

5. Why Dashboards? Data Rich Data Consolidation Information Application
Long term: Monitoring Performance (Pharmacy Historical Queue Time)
Short term: Display actionable, concise information (Pharmacy Current Wait Time)
Benefits: Automated, real-time information, available to all stakeholders
Why Dashboards: that which we measure we improve, and the importance of the shared understanding of current state business.
What is important? Why are we here? How can we boil this down to a number or ratio or statistic? What is measurable? What is actionable? Some science and some art to metrics and dashboards
Benefits: Automated and updated, viewable by anyone in the command
Other benefits: Expensive up front but cheap in the long run (FTE reduction IP dashboard)
Data Rich (Example of FTE Calculator)
Data Consolidation (Example of Clinic Dashboard)
FOR OFFICIAL USE ONLY

6. Performance Monitoring
Need for monitoring
Progress
Goal Definition
Update
Thresholds
Sustainment
Performance visibility
Clear/Simple
Audience
Need for monitoring continued: Need to define current state, measure the progress, set goals, and prepare for sustainment
Ex: Fundraisers tracking the goals, patient satisfaction
FOR OFFICIAL USE ONLY

7. Be aware of the consumers of the information
Designing a Dashboard
Key Questions before you get started:
Who is the audience?
Role
Workflow
Data skills
What value will the dashboard add?
Dashboards can provide value in a lot of different ways
What type of dashboard do we need?
Strategic? Operations? Historical? Real-time? High level? Drill-able?
Who is my audience: who is the consumer? What are their needs? What do they know? What are their prejudices? A complicating factor is that the answer is usually multiple. And they usually have difference information needs.
Who is the consumer
Role: What questions do they need answered? -> structure the information to make it super easy to answer questions
Workflow: how often and how will they be accessing the dashboard? -> typically we assume they will be using a desktop with a landscape window.
Data comfort and skills: sophistication with data? Excel? Do they enjoy the numbers? Do they know where the data comes from-> the detail should match the capabilities of their comfort zone.
Tenure: how long will the user be in their position? The need to make things user friendly and easy to understand is very important. This includes user guides and hand offs.
What value will the dashboard add
Help management define what is important
Educate people in the organization about the things that matter
Set goals and expectations for specific individuals or groups
Help executives sleep at night because they know what’s going on
Encourage specific actions in a timely manner
Highlight exceptions and provide alerts when problems occur
Communicate progress and success
Provide a common interface for interacting with and analyzing important business data
Type of dashboard:
Scope: Broad or specific? Business Role: Strategic or Operational? Time Horizon: Historical, Snapshot, Real-Time, Predictice? Level of Detail: High or Drill-able?
Be aware of the consumers of the information
FOR OFFICIAL USE ONLY

8. Metrics need to be well-understood and designed for the end user.
Designing a Dashboard
Each page or grouping of visualization needs a goal
Ask good questions
Resist the urge to add every metric available
“What if I told you…”
Reporting vs. Exploring
If the metric has not been measured or produced before, it does not belong on a dashboard
Have a goal for each page -> How can we optimize clinic schedules and provider schedules?
Ask better questions -> everything in the sink lists appear. I often ask, “what if I told you…”
Reporting vs. Exploring: if the measure has not been produced before or asked before, it does not belong on a dashboard, at least in the beginning. Metrics for dashboards need to be well-understood and predictable.
Metrics need to be well-understood and designed for the end user.
FOR OFFICIAL USE ONLY

9. Metric Selection
Key performance indicators (KPIs) allow for ongoing measurement of a system’s performance
KPIs should be:
Simple
Understood
Actionable
Credible
The perfect metric is actionable, understood, credible data, transparent and simple to calculate
Simple to calc: if you need a PHD to describe it, then its probably not good.
Understood: Referrals vs Consults. Appointments vs Visits. Utilization!!!
Actionable: Too broad – Will this information drive decisions?
Credible data: do people trust it? Is it manually entered or from a system people trust?
The perfect metric is actionable, understood, credible, transparent and simple to calculate
FOR OFFICIAL USE ONLY

10. Metric Creation Requirements and design Identify data needs
Feasibility?
Prototype metrics with team
The technology of choice will dictate the visualization tool available.
Deploy, monitor, and make revisions
Documentation!
Pitfalls: Data source changes, business process changes, too many metrics, reporting vs. exploring
Metric creation:
By time you’re at the point of developing a dashboard, the majority of the metrics should have been produced and nothing should be new.
Start with a couple of blank sheets of paper or visio or powerpoint and start with the goal of each page and write it down
Remember that people read right to left top to bottom
Must consider:
User comfort level with IT
Data availability and linkage
SharePoint
Other tools
Data sources changes (Essentris non standardized fields)
Changes in business processes can change the meaning of the dashboard
The metric creation process is iterative in nature and involves the developers, project managers, and end users
FOR OFFICIAL USE ONLY

11. Display of Metrics Types of charts Trends Pareto Stoplights Tables
Thermometers
Heat maps
Conditional Formatting
Technology and metrics
I am a big fan of using web-based dashboard tools. The technology I am most familiar with is a server based technology that leverages the Microsoft Stack. It is called MS SQL Server Reporting Services.
However, one could argue a dashboard using Excel could also equally as successful. Several other COTS products exist but I have never used them.
Functionality: Drill-downs, ed subscriptions, automated data import/exports, filters and parameters
FOR OFFICIAL USE ONLY

12. Understanding Displays
Trend charts
Displaying data over time increases the understanding of the performance of a process, especially when there is a goal.
FOR OFFICIAL USE ONLY

13. Understanding Displays
Pareto charts
Pareto with drill down
“What is the most common reason for last minute cancellations? How can I find out more information?”
Roughly 80% of the effects come from 20% of the causes
FOR OFFICIAL USE ONLY

14. Understanding Displays
Stoplights, Tables, Maps, and Conditional Formatting
FOR OFFICIAL USE ONLY

15. Setting Targets A target should be achievable Moderate Stretch
A target should be intuitive
Zero sentinel events
Lastly, metrics need a goal. Getting agreement at first can be difficult so first get buy-in on the metric then set a goal.
Some metrics are easy to get a target on: the command/bumed has policies
FOR OFFICIAL USE ONLY

16. Monitoring Using Targets
More than just meeting the target
Variability
Comparisons over time
Target: 20 minute queue time or less.
FOR OFFICIAL USE ONLY

17. Management must own the metrics – it is a top down process.
Management Decisions
Management via dashboard is enabled by:
Tool visibility at all appropriate levels
Regular review of dashboard metrics
Ownership of metrics
Identification of all metric contributors
Focus on processes and not “gaming” the metric
Don’t manage to the metric or modify data to meet the metric
You have the dashboard, the end users need to see it, meetings to review the dashboard and how do you operationalize the information into action plans.
If management has ownership, people will use it.
Management must own the metrics – it is a top down process.
FOR OFFICIAL USE ONLY

18. Case Study Data rich example of using tables to display information.
Audience: Technical and Analytical End User
Value: Provide a forecasted workload
Type: Specific, tactical, forecasting
FOR OFFICIAL USE ONLY

19. Case Study
Data consolidation example of using different types of displays.
Audience: Strategic, Director/CO Level
Value: Performance monitoring, create action items
Type: Specific, operational, historical and current
FOR OFFICIAL USE ONLY

20. Case Study
Data consolidation example of using different types of displays.
Audience: Tactical, Manager/Front line supervisor
Value: Performance monitoring, action items for today
Type: Specific, tactical, current
Tech Name 1
Tech Name 2
Tech Name 3
Tech Name 4
Tech Name 5
Tech Name 6
Tech Name ..
“How are we doing today”
FOR OFFICIAL USE ONLY

21. Questions Joe Zuchora, Analyst Improvement Path Systems
FOR OFFICIAL USE ONLY