1. Providing Treatment, Restoring Hope Program Evaluation and Improvement Using Small Tests Of Change Kristen A. Stafford, MPH Pat Bass, RN, MA Track 1.0 Meeting September 25, 2007

2. Slide 2 “Any road will do if you don’t know where you are going” - Lewis G. Carroll Alice in Wonderland

3. Slide 3

4. Slide 4

5. Slide 5 Objective So they’ve collected the data… …what now? A look at how our local partners are using data to inform and improve care and treatment

6. Slide 6 Components of our Process Patient OutcomesData Quality Care Delivery System  S T O C

7. Slide 7 What is the STOC Process  See What is your data telling you What is your goal Try/Track What will you try? Who will do it? When How will you track? Observe What happened? Did it work? What were the balancing results? Continue Do it on a larger scale or try something else.

8. Slide 8 What is it  Intended to speed up system improvements  Evidenced based management  Site driven  Incorporated into daily routine  Consistent and repeated reviews of information already captured

9. Slide 9 Why do it  To build the monitoring and evaluation capacity of in-country teams and treatment sites to provide a sustainable system of quality care and treatment  To work ourselves out of a job

10. Slide 10 Steps to STOC  Leadership buy in  Engage all levels of care delivery and clinic teams  Incorporate STOC activities into day to day duties – not extra work  Try something small  If it works, keep doing it…if it doesn’t try something else  Keep track of what you try and what happened  Share, share, share now

11. Slide 11 How to tell if its an improvement % with undetectable Viral LoadMonths Started adherence support group on clinic days Goal/target

12. Slide 12 Patient Outcomes  On treatment cross-sectional review  Randomized population of focus Started on tx 9 – 15 months before review  Chart abstraction  Patient Survey  Viral Load  Aggregated and by treatment site analysis  Group and site by site feedback  Selection of indicators most related to viral suppression and failure  STOC development with sites  Bi-annual to annual

13. Slide 13 Patient Outcome Tools

14. Slide 14 Example of Findings Statistically Significant p<.05

15. Slide 15 Small Test of Change Example

16. Slide 16 Building Local Capacity: ARV Pickup Tracking ARV pickup in Kenya  Goal: To improve patient adherence to picking up ARVs  Strategy: Automated reports were created in IQTools: 1) Identify patients that are supposed to come in during a certain time period (i.e. this week) to pick up ARVs 2) Identify patients that have missed picking up their ARVs after x days Reports are run and analyzed by LPTF  LPTFs are required to report (monthly) the number of patients that did not pick up their ARVs within x days and feedback is shared

17. Slide 17 Pharmacy Visits

18. Slide 18 How can LPTF use the data?  Based on the data, the local Clinical, SI, and Management Team work with LPTFs to develop a small test of change process to improve patient adherence to picking up ARVs  Since this process was implemented, LPTFs have had positive outcomes in the number of patients picking up their ARVs within x days  Best practices shared across project

19. Slide 19 STOC Plan from LPTFs  The monthly analysis of missed ARV appointments can point to problems in various areas: Data entry backlog/data entry errors Poor communication True defaulters

20. Slide 20 Data entry backlog/data entry errors Problem Defined: Computerizing patient data is long and data entry errors leading to patients erroneously appearing as defaulters. One STOC Plan (Mombasa):  76 patients late in May but only 9 by June; patients in May were not true defaulters. The problem was due to an accumulation of backlog  Put a plan in place to use additional temporary data clerks to clear their backlog. [Temp staff must be trained in PMM system]  Eliminated backlog and true default rate determined  Implemented as solution when/if backlog arises

21. Slide 21 Data Quality  Scalable, robust, flexible and sustainable platforms for data collection and retrieval  Routine data cleaning and monitoring  Treatment site driven data use and analysis for adaptive management

22. Slide 22 Patient Monitoring and Management Systems: IQ Solutions Strategy:  To offer a library of tools and solutions built around adaptive management, quality, and sustainability  Requirements developed through practical field experience and lessons learned  Collaborative approach using local experts throughout the development process  Current areas of focus PMM, ART Registers, Data Quality Tools

23. Slide 23 Summary  Quality evaluation and improvement activities are a vital function of program management  The ability to scale ARV treatment programs ultimately will be dependent on efficient and sustainable care.  Sustainable care is intimately tied to achieving consistent high levels of medical care.  Data is for more than data reporting  Changing the culture of data capture and use will ensure the most effective and sustainable use of this funding

24. Slide 24 Thank You