Overview The Importance of Data: As easy as balancing your checkbook

(Title) Name(s) of presenter(s) Organizational Affiliation What do you need to know? 2 Categories – Deposits – Expenditures

How does this knowledge help? Deposits Number of Deposits Total Amount In Expenditures Number of Checks Written Total Amount Out

What can you do with the knowledge? Net Profit = Deposits - Expenditures Hit Jackpot Broke the Bank

What can you do with the knowledge? Ask Questions – What type of expenditures? – When do they occur? – Why did I experience a loss? Profit? – What happened that was different this month?

What is the rest of the story? 1. Deposits 2. Expenditures 1. Total Deposits 2. Number of Deposits 3. Total Expenditures 4. Number of Expenditures 5. Net Profit or Loss

Power of Two

How does that translate to measuring the impact of change? 1. Number of First Contacts 2. Number of Assessments 3. Elapsed Time 4. Assessment Conversion 1. Date of First Contact 2. Date of Assessment

How does that translate to measuring the impact of change? 1. Number of Clients w/First Treatment Session 2. Number of Clients w/Fourth Treatment Session 3. Elapsed Time between First and Fourth Treatment Sessions 4. % of Clients w/1 st Treatment Session who get a 4 th Treatment Session 1. Date of First Treatment Session 2. Date of Fourth Treatment Session

How does that translate to measuring the impact of change? 1.Three Counts 2.Two Conversion Numbers 3.Three Elapsed Time Figures 1. Date of First Contact 2. Date of Assessment 3. Date of Admissions

WHY IS THIS IMPORTANT?

Principle #5 Rapid Cycle Testing Start by asking 3 questions – What are we trying to accomplish? – How will we know the change is an improvement? – What changes can we test that will result in an improvement? Model for Improvement Reference: Langley, Nolan, Nolan, Norman, & Provost. The Improvement Guide, San Francisco, Jossey-Bass Publishers, 1996

7 Simple Rules of the Road Define measures Collect baseline data Establish a clear aim Consistent collection Avoid common pitfalls Report and Chart progress Ask questions

– Use of Data 2X2 Tables Pre Request to Service Post Request to Service Pre Admissions Post Admissions

Rule 1: Define Measures Establish clear definitions Clarify project aims Agreed upon by key stakeholders

Establishing Clear Definitions Example suggested measures – Time from First Contact to Assessment – # of clients moving to the next level of care – % of clients attending their 1 st four sessions Measure definition – Elapsed Time from Date of Assessment – Date of 1 st Contact – # of admissions to next LOC/# of discharges from higher LOC – # of clients with four sessions/# of admissions

Rule 2: Establish a Baseline Never start a project without it Define a clear starting point Use agreed-upon definition StartFinish

Baseline Data Serve as a meaningful “road map” Helps answer the question: “How will we know a change is an improvement?” Use tools to collect a adequate baseline information The time period for the baseline will vary by – Measure – Agency’s size – Ease of Collection Preferred Sample Size is at least 40

Examples of Baseline Data Average time from 1 st Contact to Assessment is 35 Days 40% of our assessments are no-shows Only 30% of our clients receive 4 units of service in 30 days Only 25% of Detoxification Discharges connect with the next level of care

Rule 3: Establish an aim Be flexible – Information suggests changing the aim, change it – Aim is too ambitious, set a realistic aim that still challenges the agency to improve – Aim is easily achieved, set a more ambitious aim that stretches the agency’s capacity to improve

Examples – Defining the Aims No: ↑ % of Res Discharges to OP by 20% Yes: ↑ Successful Res Discharges to OP from 40 to 48% No: ↓ Assessment No-Shows by 40% Yes: ↓ Assessment No-Shows from 60 to 36% No: ↓ Time from 1 st Contact to Assessment by 20 days Yes: ↓ Time from 1st Contact to Assessment from 28 to 7 days

Rule 4: Consistently Collect Data The length of time necessary to test a change will vary depending on an agency’s size. Devise ways to collect information but remember KEEP IT SIMPLE Collect small samples over short time periods – Next 10 clients – Next 15 Phone Calls – Measure impact in days not weeks or months – Preferred Sample Size is at least 40 – Pilots should not last more than a month Once change is successful, collect larger samples to verify progress – Track clients admitted next week – Examine data for one month

Rule 5: Avoid Common Pitfalls 1.Events occurring before their time 2.Sequential events occurring in order but with long lag times 3.Sequential events occurring out of order 4.Missing dates 5.Cell contains characters 6.Incorrect recording of dates

Dates before their time Dates out of sequence Long lag times

Stay out of the Quicksand Don’t collect to much data Don’t focus on too many measures Don’t get trapped in analysis paralysis Ask a “yes or no” question of your data

Rule 6: Report and Chart Progress A Simple Axiom: One chart, one message Charts can be used to: – Highlight the baseline (pre-change) data – Identify when a change was introduced – Visually represent the impact of individual changes over time, and – Inform your agency about sustaining change over time.

Example Chart

Use Trendlines

Rule 7: Ask Questions Do not accept results at face value Do the results look right? What is the data telling us? Unsuccessful changes afford the opportunity to ask Why?

What do you know with more information? (e.g. Gender, Age, Drug of Choice) 1. Number of First Contacts 2. Number of Assessments 3. Elapsed Time 4. Assessment Conversion 1. Date of First Contact 2. Date of Assessment

Summary Determine what you will measure Establish a system to collect key data elements Gather your baseline Define your aim Collect data often and consistently – who’s job is this?! Evaluate the impact – One chart, one message – Ask questions

QUESTIONS