Monitoring and Reporting Performance Metrics Improvement Path Systems – Joe Zuchora

Context, Purpose, Outcome 1) Context: Navy Medicine can benefit from monitoring and reporting performance metrics to help drive better decisions at all management levels. 2) Purpose: Guidance will be provided in designing and creating dashboards, and examples of successful dashboards used for performance monitoring will be discussed 3) Outcome: Identification of the need for dashboards and methodology for creating and using dashboards. FOR OFFICIAL USE ONLY

Agenda What is a Dashboard? Why Dashboards? Monitoring of performance Dashboard design Metric selection and creation Display of metrics Setting targets Management decisions Case study FOR OFFICIAL USE ONLY

What is a Dashboard? -Peter McFadden, CEO Excel Dashboard Widgets “An easy to read, often single page, real-time user interface, showing a graphical presentation of the current status and historical trends of an organization’s Key Performance Indicators” -Peter McFadden, CEO Excel Dashboard Widgets FOR OFFICIAL USE ONLY

Why Dashboards? Data Rich Data Consolidation Information Application Long term: Monitoring Performance (Pharmacy Historical Queue Time) Short term: Display actionable, concise information (Pharmacy Current Wait Time) Benefits: Automated, real-time information, available to all stakeholders Why Dashboards: that which we measure we improve, and the importance of the shared understanding of current state business. What is important? Why are we here? How can we boil this down to a number or ratio or statistic? What is measurable? What is actionable? Some science and some art to metrics and dashboards Benefits: Automated and updated, viewable by anyone in the command Other benefits: Expensive up front but cheap in the long run (FTE reduction IP dashboard) Data Rich (Example of FTE Calculator) Data Consolidation (Example of Clinic Dashboard) FOR OFFICIAL USE ONLY

Performance Monitoring Need for monitoring Progress Goal Definition Update Thresholds Sustainment Performance visibility Clear/Simple Audience Need for monitoring continued: Need to define current state, measure the progress, set goals, and prepare for sustainment Ex: Fundraisers tracking the goals, patient satisfaction FOR OFFICIAL USE ONLY

Be aware of the consumers of the information Designing a Dashboard Key Questions before you get started: Who is the audience? Role Workflow Data skills What value will the dashboard add? Dashboards can provide value in a lot of different ways What type of dashboard do we need? Strategic? Operations? Historical? Real-time? High level? Drill-able? Who is my audience: who is the consumer? What are their needs? What do they know? What are their prejudices? A complicating factor is that the answer is usually multiple. And they usually have difference information needs. Who is the consumer Role: What questions do they need answered? -> structure the information to make it super easy to answer questions Workflow: how often and how will they be accessing the dashboard? -> typically we assume they will be using a desktop with a landscape window. Data comfort and skills: sophistication with data? Excel? Do they enjoy the numbers? Do they know where the data comes from-> the detail should match the capabilities of their comfort zone. Tenure: how long will the user be in their position? The need to make things user friendly and easy to understand is very important. This includes user guides and hand offs. What value will the dashboard add Help management define what is important Educate people in the organization about the things that matter Set goals and expectations for specific individuals or groups Help executives sleep at night because they know what’s going on Encourage specific actions in a timely manner Highlight exceptions and provide alerts when problems occur Communicate progress and success Provide a common interface for interacting with and analyzing important business data Type of dashboard: Scope: Broad or specific? Business Role: Strategic or Operational? Time Horizon: Historical, Snapshot, Real-Time, Predictice? Level of Detail: High or Drill-able? Be aware of the consumers of the information FOR OFFICIAL USE ONLY

Metrics need to be well-understood and designed for the end user. Designing a Dashboard Each page or grouping of visualization needs a goal Ask good questions Resist the urge to add every metric available “What if I told you…” Reporting vs. Exploring If the metric has not been measured or produced before, it does not belong on a dashboard Have a goal for each page -> How can we optimize clinic schedules and provider schedules? Ask better questions -> everything in the sink lists appear. I often ask, “what if I told you…” Reporting vs. Exploring: if the measure has not been produced before or asked before, it does not belong on a dashboard, at least in the beginning. Metrics for dashboards need to be well-understood and predictable. Metrics need to be well-understood and designed for the end user. FOR OFFICIAL USE ONLY

Metric Selection Key performance indicators (KPIs) allow for ongoing measurement of a system’s performance KPIs should be: Simple Understood Actionable Credible The perfect metric is actionable, understood, credible data, transparent and simple to calculate Simple to calc: if you need a PHD to describe it, then its probably not good. Understood: Referrals vs Consults. Appointments vs Visits. Utilization!!! Actionable: Too broad – Will this information drive decisions? Credible data: do people trust it? Is it manually entered or from a system people trust? The perfect metric is actionable, understood, credible, transparent and simple to calculate FOR OFFICIAL USE ONLY

Metric Creation Requirements and design Identify data needs Feasibility? Prototype metrics with team The technology of choice will dictate the visualization tool available. Deploy, monitor, and make revisions Documentation! Pitfalls: Data source changes, business process changes, too many metrics, reporting vs. exploring Metric creation: By time you’re at the point of developing a dashboard, the majority of the metrics should have been produced and nothing should be new. Start with a couple of blank sheets of paper or visio or powerpoint and start with the goal of each page and write it down Remember that people read right to left top to bottom Must consider: User comfort level with IT Data availability and linkage SharePoint Other tools Data sources changes (Essentris non standardized fields) Changes in business processes can change the meaning of the dashboard The metric creation process is iterative in nature and involves the developers, project managers, and end users FOR OFFICIAL USE ONLY

Display of Metrics Types of charts Trends Pareto Stoplights Tables Thermometers Heat maps Conditional Formatting Technology and metrics I am a big fan of using web-based dashboard tools. The technology I am most familiar with is a server based technology that leverages the Microsoft Stack. It is called MS SQL Server Reporting Services. However, one could argue a dashboard using Excel could also equally as successful. Several other COTS products exist but I have never used them. Functionality: Drill-downs, emailed subscriptions, automated data import/exports, filters and parameters FOR OFFICIAL USE ONLY

Understanding Displays Trend charts Displaying data over time increases the understanding of the performance of a process, especially when there is a goal. FOR OFFICIAL USE ONLY

Understanding Displays Pareto charts Pareto with drill down “What is the most common reason for last minute cancellations? How can I find out more information?” Roughly 80% of the effects come from 20% of the causes FOR OFFICIAL USE ONLY

Understanding Displays Stoplights, Tables, Maps, and Conditional Formatting FOR OFFICIAL USE ONLY

Setting Targets A target should be achievable Moderate Stretch A target should be intuitive Zero sentinel events Lastly, metrics need a goal. Getting agreement at first can be difficult so first get buy-in on the metric then set a goal. Some metrics are easy to get a target on: the command/bumed has policies FOR OFFICIAL USE ONLY

Monitoring Using Targets More than just meeting the target Variability Comparisons over time Target: 20 minute queue time or less. FOR OFFICIAL USE ONLY

Management must own the metrics – it is a top down process. Management Decisions Management via dashboard is enabled by: Tool visibility at all appropriate levels Regular review of dashboard metrics Ownership of metrics Identification of all metric contributors Focus on processes and not “gaming” the metric Don’t manage to the metric or modify data to meet the metric You have the dashboard, the end users need to see it, meetings to review the dashboard and how do you operationalize the information into action plans. If management has ownership, people will use it. Management must own the metrics – it is a top down process. FOR OFFICIAL USE ONLY

Case Study Data rich example of using tables to display information. Audience: Technical and Analytical End User Value: Provide a forecasted workload Type: Specific, tactical, forecasting FOR OFFICIAL USE ONLY

Case Study Data consolidation example of using different types of displays. Audience: Strategic, Director/CO Level Value: Performance monitoring, create action items Type: Specific, operational, historical and current FOR OFFICIAL USE ONLY

Case Study Data consolidation example of using different types of displays. Audience: Tactical, Manager/Front line supervisor Value: Performance monitoring, action items for today Type: Specific, tactical, current Tech Name 1 Tech Name 2 Tech Name 3 Tech Name 4 Tech Name 5 Tech Name 6 Tech Name .. “How are we doing today” FOR OFFICIAL USE ONLY

Questions Joe Zuchora, Analyst Improvement Path Systems zuchora@improvmentpath.com 248-935-6364 FOR OFFICIAL USE ONLY