1 Measuring & Managing Health Services: The Balanced Scorecard David Peters, Director, Future Health Systems Research Consortium Sept 28, 2006

2 What gets measured gets managed. What gets managed gets done. Tom Peters

3 Would This Measurement Describe Your Child?

4 Why Have a Balanced Scorecard? Health sector is complex, has many components Need an efficient way to assess multiple objectives Overloaded with different types of reports Stakeholders demand vigilance Poor measurement can lead to crisis

5 Balanced Scorecard – Original Definition A multi-dimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking objectives, initiatives, and measures to an organizations strategy. Kaplan & Norton, 1996

6 Scorecard Domains Which dimensions should we measure?

7 Balanced Scorecard: General Domains Customer Results Internal Processes Staff and Organizational Growth Financial Results Vision & Strategy

8 Qualities of Good Scorecard Measures Valid Reliable Balanced Easily Understood Intermediable Agreed Upon Limited Specific Measurable Achievable Targets Relevant Timely

9 Presenting Scorecards: Performance Dashboards Simple to see & interpret Multi-dimensional measures of critical indicators of overall performance Allows for the early detection of problems and successes Can be benchmarked against other data

10 Afghanistan National Health Services Performance Assessment (NHSPA)

11 Developing the Balanced Scorecard Frontline providers, NGOs, MOPH, donors to agree on: Purpose of BSC Domains to measure Unit of analysis Process & frequency of review/decisions Principles for benchmarking Short-listing indicators based on face validity, importance, reliability

12 Afghanistan Balanced Scorecard Domains 1.Patient & Community Results 2.Staff Results 3.Capacity for Service Provision 4.Service Provision Results 5.Financial System Results 6.Overall Vision Results

13 Measuring perceptions Many survey questions sought perceptions. But how to ask? Likert scale couldnt be understood Money? ExcellentGoodFair Poor The Naanogram

NHSPA 2005 NHSPA Provinces3330 Facilities Basic Health Centers Comprehensive HCs District Hospitals Observations Exit Interviews Health worker Interviews CHW Interviews Afghanistan Sample for Balanced Scorecard

Afghanistan NHSPA Provincial Balanced Scorecard

16 Mean Score (29 Provinces) Difference 1. Overall patient satisfaction Patient perceptions of quality Written Shura- e-Sehie Activities Patients & Community Perspectives

17 Mean Score (29 Provinces) Difference 4. Health worker satisfaction Salary payments current Staff Perspectives

18 Mean Score (29 Provinces) Difference 6. Equipment functionality Drug availability Family Planning availability Laboratory functionality Meeting minimum staff guidelines Capacity for Service Provision

19 Mean Score (29 Provinces) Difference 11. Provider knowledge Staff training HMIS use index Clinical guidelines Infrastructure index Capacity for Service Provision

20 Mean Score (29 Provinces) Difference 18. Patient history and physical exam Patient counseling Proper sharps disposal New outpatient visits (>750/month) Service Provision

21 Mean Score (29 Provinces) Difference 22. Time spent with patient (>9 minutes) Provision of ANC Provision of delivery care Service Provision

22 Mean Score (29 Provinces) Difference 25. User fee guidelines Exemptions for poor patients Financial Systems

23 Mean Score (29 Provinces) Difference 27. Females as % of new outpatients Outpatient visit concentration index Pt. satisfaction concentration index Overall Vision

24 Identifying Exceptional Provincial Performance Top Performers Change Wardak18.0 Balkh16.0 Ghor15.5 Faryab12.0 Saripul11.1 Bottom Performers Change Herat-2.2 Paktika-3.0 Nangarhar-4.7 Nimroz-5.5 Nuristan-13.7

25 Balanced Scorecard: Improvements Progress has been made in most areas Improvements in 22 out of 29 indicators Areas of largest improvement include: Shura-e-Sehie activity Salary payments current Laboratory functionality Drug availability Meeting minimum staffing guidelines

26 Balanced Scorecard: Problems Improvement not consistent across all provinces or aspects of service delivery Areas of concern: Time spent with patients Facility infrastructure Sharps disposal User fee guidelines Patient records

27 Balanced Scorecard on Contract Performance: Cumulative Ratings Difference Performance Based Contracts (WB) Contracts with Technical Assistance (USAID) Ministry of Health Strengthening (WB) Contracting Without Performance (EU) Usual Management

28 Some Final Thoughts

29 Successful Scorecards Focus on significant, success- determining measures of organizations total performance Based on timely information Predictive (leading) indicators more useful than trailing indicators

30 You cant fatten a cow by weighing it

31 Other Examples

32 Balanced Scorecard At One Hospital Satisfaction Quality of Worklife Social Accountability Growth Quality Financial & Operational

33 Voluntary Hospital Association TICU Collaborative Seven domains Clinical performance Leadership Technology and design Process management Financial management Workforce Patient and family experience TICU = Transformation of the Intensive Care Unit

34 Source: VHA, Inc. Used with Permission