1. 1 Measuring & Managing Health Services: The Balanced Scorecard David Peters, Director, Future Health Systems Research Consortium Sept 28, 2006

2. 2 What gets measured gets managed. What gets managed gets done. Tom Peters

3. 3 Would This Measurement Describe Your Child?

4. 4 Why Have a Balanced Scorecard? Health sector is complex, has many components Need an efficient way to assess multiple objectives Overloaded with different types of reports Stakeholders demand vigilance Poor measurement can lead to crisis

5. 5 Balanced Scorecard – Original Definition A multi-dimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking objectives, initiatives, and measures to an organizations strategy. Kaplan & Norton, 1996

6. 6 Scorecard Domains Which dimensions should we measure?

7. 7 Balanced Scorecard: General Domains Customer Results Internal Processes Staff and Organizational Growth Financial Results Vision & Strategy

8. 8 Qualities of Good Scorecard Measures Valid Reliable Balanced Easily Understood Intermediable Agreed Upon Limited Specific Measurable Achievable Targets Relevant Timely

9. 9 Presenting Scorecards: Performance Dashboards Simple to see & interpret Multi-dimensional measures of critical indicators of overall performance Allows for the early detection of problems and successes Can be benchmarked against other data

10. 10 Afghanistan National Health Services Performance Assessment (NHSPA)

11. 11 Developing the Balanced Scorecard Frontline providers, NGOs, MOPH, donors to agree on: Purpose of BSC Domains to measure Unit of analysis Process & frequency of review/decisions Principles for benchmarking Short-listing indicators based on face validity, importance, reliability

12. 12 Afghanistan Balanced Scorecard Domains 1.Patient & Community Results 2.Staff Results 3.Capacity for Service Provision 4.Service Provision Results 5.Financial System Results 6.Overall Vision Results

13. 13 Measuring perceptions Many survey questions sought perceptions. But how to ask? Likert scale couldnt be understood Money? ExcellentGoodFair Poor The Naanogram

14. 14 2004 NHSPA 2005 NHSPA Provinces3330 Facilities 617 628 Basic Health Centers 323 368 Comprehensive HCs 243 217 District Hospitals 51 43 Observations 5719 5863 Exit Interviews 5597 5863 Health worker Interviews 1553 1458 CHW Interviews 167 306 Afghanistan Sample for Balanced Scorecard

15. Afghanistan NHSPA Provincial Balanced Scorecard

16. 16 Mean Score (29 Provinces) 20042005Difference 1. Overall patient satisfaction 81.384.12.8 2. Patient perceptions of quality 74.176.52.4 3. Written Shura- e-Sehie Activities 39.952.412.5 Patients & Community Perspectives

17. 17 Mean Score (29 Provinces) 20042005Difference 4. Health worker satisfaction 61.964.02.1 5. Salary payments current 68.983.915.0 Staff Perspectives

18. 18 Mean Score (29 Provinces) 20042005Difference 6. Equipment functionality 65.266.31.1 7. Drug availability65.382.216.9 8. Family Planning availability 60.265.65.3 9. Laboratory functionality 17.639.321.7 10. Meeting minimum staff guidelines 32.552.820.3 Capacity for Service Provision

19. 19 Mean Score (29 Provinces) 20042005Difference 11. Provider knowledge 55.168.613.5 12. Staff training39.271.932.7 13. HMIS use index67.161.1-6.0 14. Clinical guidelines 35.448.413.0 15. Infrastructure index 54.344.1-10.2 Capacity for Service Provision

20. 20 Mean Score (29 Provinces) 20042005Difference 18. Patient history and physical exam 69.574.75.1 19. Patient counseling 36.738.11.4 20. Proper sharps disposal 58.548.9-9.6 21. New outpatient visits (>750/month) 27.642.815.2 Service Provision

21. 21 Mean Score (29 Provinces) 20042005Difference 22. Time spent with patient (>9 minutes) 19.810.1-9.6 23. Provision of ANC56.170.114.0 24. Provision of delivery care 24.526.41.9 Service Provision

22. 22 Mean Score (29 Provinces) 20042005Difference 25. User fee guidelines 84.976.5-8.4 26. Exemptions for poor patients 78.985.16.2 Financial Systems

23. 23 Mean Score (29 Provinces) 20042005Difference 27. Females as % of new outpatients 53.756.32.6 28. Outpatient visit concentration index -0.008-0.021-0.013 29. Pt. satisfaction concentration index -0.0010.0040.005 Overall Vision

24. 24 Identifying Exceptional Provincial Performance Top Performers Change 2004-05 Wardak18.0 Balkh16.0 Ghor15.5 Faryab12.0 Saripul11.1 Bottom Performers Change 2004-05 Herat-2.2 Paktika-3.0 Nangarhar-4.7 Nimroz-5.5 Nuristan-13.7

25. 25 Balanced Scorecard: Improvements Progress has been made in most areas Improvements in 22 out of 29 indicators Areas of largest improvement include: Shura-e-Sehie activity Salary payments current Laboratory functionality Drug availability Meeting minimum staffing guidelines

26. 26 Balanced Scorecard: Problems Improvement not consistent across all provinces or aspects of service delivery Areas of concern: Time spent with patients Facility infrastructure Sharps disposal User fee guidelines Patient records

27. 27 Balanced Scorecard on Contract Performance: Cumulative Ratings 20042005Difference Performance Based Contracts (WB) 52.364.412.1 Contracts with Technical Assistance (USAID) 53.463.610.2 Ministry of Health Strengthening (WB) 53.761.88.1 Contracting Without Performance (EU) 54.257.73.5 Usual Management51.548.5-3.0

28. 28 Some Final Thoughts

29. 29 Successful Scorecards Focus on significant, success- determining measures of organizations total performance Based on timely information Predictive (leading) indicators more useful than trailing indicators

30. 30 You cant fatten a cow by weighing it

31. 31 Other Examples

32. 32 Balanced Scorecard At One Hospital Satisfaction Quality of Worklife Social Accountability Growth Quality Financial & Operational

33. 33 Voluntary Hospital Association TICU Collaborative Seven domains Clinical performance Leadership Technology and design Process management Financial management Workforce Patient and family experience TICU = Transformation of the Intensive Care Unit

34. 34 Source: VHA, Inc. Used with Permission