1. Part I – Data Collection and Measurement Ruth S. Gubernick, MPH Quality Improvement Advisor Lori Morawski, MPH CHES Manager, Quality Improvement Programs Florida Pediatric Medical Home Demonstration Project Learning Session I September 23-24, 2011

2. Disclosure We have no relevant financial relationships with the manufacturers of any commercial products and/or provider of commercial services discussed in this CME activity. We do not intend to discuss an unapproved/investigative use of a commercial product/device in their presentation.

3. Measurement, Data Collection & Evidence of Change  Importance of measurement – Why?  How will we know that a change is an improvement?  Clarify and be directly linked to goals  Seek usefulness over perfection  Be integrated into daily work whenever possible  Be graphically and visibly displayed  For PDSA cycles, be simple and feasible enough to accomplish in close time proximity to tests of change

4. How will we know that a change is an improvement? Plan DoStudy Act What are we trying to accomplish? What changes can we make that will result in improvement? The Improvement Model The Improvement Guide Associates in Process Improvement

5. Measurement for Quality Improvement  You can’t improve what you can’t (or don’t) measure  A good aim statement provides clear direction.  Measures tell a team if the changes they make are making a difference  Measurement tells you where you are and where you are going

6. Measures  Need to define  Target population  Numerator  Denominator

7. Florida Pediatric Medical Home Demonstration Project: Example  Target Population:  All Medicaid/KidCare infants/children seen by clinicians in participating practice  Numerator:  Total number of patients whose individual and family concerns are elicited at this visit  Denominator:  All Medicaid/KidCare infants/children seen in participating practice whose charts are reviewed during the month of interest.

8. Effective Measurement  Seek usefulness, not perfection  Keep measurement simple, think big, but start small

9. Effective Measurement: Outcomes Outcome measures: represents the voice of the customer or patient  Hospitalizations or ED visits due to asthma  Hospital readmissions w/in 30 days due to asthma  Patient satisfaction with time it takes to schedule an appointment

10. Effective Measurement: Processes  Process measures: represents the workings of the system  Percent of patients with all expressed concerns addressed or with plans made to address them  Percent of patients who have a medical summary or comprehensive care plan created or updated/maintained at this visit

11. Effective Measurement  Build measurement into daily work routine  Data should be easy to obtain and timely  Small samples over time  Use quantitative and qualitative data  Quantitative data is highly informative  Qualitative data is easy to obtain

12. Why Plot Data Over Time  You develop a process for patients/families to have a current copy of their medical summary or comprehensive care plan reviewed and offered to them at their visit. The 6 months before implementing the process the average % of patients/families having a current copy of their medical summary or comprehensive care plan reviewed and offered to them is 10%. Six months after the process is implemented, the average % is 90%.  How will you answer the question: was this change an improvement?

13. Change Run Charts Aug. 2011 September October November December Jan. 2012 February March

14. Change Aug. 2011 September October November December Jan. 2012 February March

15. Data Collection Web Site www.aap.org/qualityimprovement/quiin/workspaces/MedHome/ DataCollection.html www.aap.org/qualityimprovement/quiin/workspaces/MedHome/ DataCollection.html

16. Requirements of Teams  From October 15, 2011 through March 30, 2012 you will be asked to submit a total of 10 patient chart reviews using the EQIPP.  Data cycles will be open on the 15th of each month. Please do not submit data until the 15th of every month. Data will be due by the 30th of each month.  Your practice will have the ability to close your own data cycle once you have submitted 10 total chart reviews each month.  If you do not have 10 charts to review in a particular month, please contact project staff and they will manually close your data cycle for you.  You will need to complete the on-line chart review survey for each patient chart you review. You have the option of first completing a “hardcopy” survey of each chart reviewed, using the PDF Data Collection Tool, but you must then submit that data using EQIPP.

17. Data Collection Tool

18. Chart Review Log Sheet Date of visit Log Number (“Patient Code”) Patient Name Medical Record or other ID # Medicaid, Medicaid health plan or KidCare ID’d as CYSHCN 001 002 003 004 005 006 007 008 009 010

19. EQIPP Data Input

20. EQIPP Data User View Each individual user can analyze their own results real time!

21. Data Reporting  QI Advisor and AAP staff will run monthly reports to share with participating practices:  Project measure reports  Measure Reports (all practices)  Practice Reports (all measures)

22. Data Explorer and Reporting

23. Reports  Reports include comprehensive data for all practices – provide more information than EQIPP  Reports will be shared monthly prior to monthly calls  Reports will be posted on project workspace  Monthly calls:  3 rd Wednesday of the month  2 nd Tuesday of the month

24. Monthly Progress Provides information about  Tests of change completed and tools used each month  Assessment of team progress  Other qualitative measures Instructions  Insert your practice’s Aim statement.  Indicate the change package items you have tested. Describe specific changes (by domain) and tools you have tested.  Rate your team’s progress using the scale. Report your team’s learning.  Complete the Systems Index.  Submit your Monthly Progress Report by the 30 th of the month for which you are submitting, using the Survey Monkey link (URL) that will be sent each month via the project listserv

25. Example Chart from Monthly Progress Report Index Aug. 2011 Sept Oct. Nov. Dec Jan. 2012 Feb. March

26. Project Workspace  All data and run charts will be available for review on the Project Workspace Web site  http://www.aap.org/qualityimprovement/quiin/workspaces/ http://www.aap.org/qualityimprovement/quiin/workspaces/ MedHome/Home.html  Compare your team’s data to other teams and to the aggregate data!

27. Data Collection Cycle Table Data Cycle Table Data Cycle LabelMonth of Data Cycle Data Cycle OpensData Cycle ClosesData to be submitted BaselineAugustAugust 15, 2011August 30, 20111 st 20 chart of Medicaid, Medicaid health plan or KidCare patients seen in August Follow-up 1OctoberOctober 15, 2011October 30, 20111 st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in October Follow-up 2NovemberNovember 15, 2011November 30, 20111 st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in November Follow-up 3DecemberDecember 15, 2011December 30, 20111 st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in December Follow-up 4JanuaryJanuary 15, 2012January 30, 20121 st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in January Follow-up 5FebruaryFebruary 15, 2012February 28, 20121 st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in February Follow-up 6MarchMarch 15, 2012March 30, 20121 st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in March Teams should not submit data until the Data Cycle Opens (15 th of month) Note: teams can close their own data cycle after submitting 10 charts. If they do not have 10 charts in a particular month, staff will close the data cycle for them.

28. Questions