1. 1 Using Data to Drive Health System Performance Commissioned from Ovations by the National Primary and Care Trust Development Programme

2. 2 Health Care Evaluation & Measurement for PCTs Some Key Evaluation Terms Acceptable Accessible Accomplishment Accountable Accuracy Analysis Appropriate Available Cohorts Comparison Content Context Control Cost Data: primary and secondary Goals Judgment Metrics Norms Objectives Outcomes Outputs Precision Process Purpose Quality Quantity Recording Reliability Reporting Standards Synthesis Time Timelines Validity Value Weighting Worth

3. 3 Health Care Evaluation & Measurement for PCTs Things to Consider Comparing and contrasting various types of evaluation Identification of problems that may hinder an effective evaluation Reasons to conduct an evaluation The planning and conducting of an evaluation Defining metrics and methods to measure and report evaluation findings

4. 4 Health Care Evaluation Key Ideas for PCTs Evaluation is critical for all health care, and prevention/ health promotion initiatives. Evaluation must be designed early in the process of health care planning. Remember PRECEDE/PROCEED! The process of designing an evaluation must be a collaborative effort of all stakeholders. Evaluation ultimately becomes judgment. Who has the power to decide? Evaluation does not need to be a formal academic study Do you report successes, failures, strengths, weaknesses?

5. 5 Health Care Evaluation Purposes for PCTs 1. To determine achievement of objectives related to improved health status 2. To improve health program implementation 3. To provide accountability to funders 4. To increase community support 5. To contribute to the scientific base for community. public health interventions 6. To impact policy decisions

6. 6 Health Care Evaluation Definition Evaluation: Determining the value or worth of the health care initiative against a standard of acceptability. To examine or judge. (The key is who establishes the standard and who judges!)

7. 7 Health Care Evaluation Elements Context: What, when, where, and who Process: How care is organized and delivered Content: Program elements to be provided and why - available birdseed Output: How many times did the bird flap its wings? Outcome: Did the bird fly? Impact: How high? How far? Where?

8. 8 Health Care Evaluation Types Process Evaluation: Examines the procedures and tasks involved during the implementation of a program.

9. 9 Health Care Evaluation Types Process Evaluation When to use: As soon as the health initiative begins What it shows: How well a program is working as it goes Why is it useful: Identifies early problems

10. 10 Health Care Evaluation Types Outcome Evaluation: Used to obtain descriptive data on a project and to document short-term results. Focuses on an ultimate goal of a health care program or treatment. Generally measured by vital statistics in a population.

11. 11 Health Care Evaluation Types Outcome Evaluation When to use: For ongoing programs at appropriate intervals or for one time programs when program is complete What it shows: Has program reached its ultimate goal. Why is it useful: Learn from successes and for future funding.

12. 12 Health Care Evaluation Types Impact Evaluation: Is the most comprehensive type of evaluation because it focuses on the long- range results and the resultant improvements in health status. Impact evaluation is the most costly. Information obtained from an impact evaluation can include changes in e.g.,morbidity and mortality.

13. 13 Health Care Evaluation Types Impact Evaluation When to use: After the health program has made contact with at least one person or a population What it shows: Changes in knowledge, attitudes, and beliefs Why is it useful: Allows management to modify resources effectively

14. 14 Health Care Evaluation Lets pick a population health programme idea for your specific PCT and consider aspects of process, outcome and impact evaluation.

15. 15 Metrics and Methods to Measure and Report Indicators of Hospital Utilisation Assumption: >Hospital admission is an adverse event that should be avoided whenever possible >Not all admissions are preventable Two major indicators >Admissions per 1000 >Days per 1000

16. 16 Metrics and Methods to Measure and Report Indicators of Hospital Utilisation Admissions per 1000 > In part measures a systems success at preventing acute disease episodes or caring for them in alternative systems > Considered a prevention indicator Days per 1000 > Reflects both frequency of admission and length of stay > Measure episode characteristics

17. 17 Metrics and Methods to Measure and Report Indicators of Hospital Utilisation Both measures are typically collected separately for: > Adult and pediatric medical > Surgical > ICU > Obstetrics > Boarder babies (newborn days after mothers discharge) > Mental health > Substance abuse

18. 18 Metrics and Methods to Measure and Report Risk Adjustment Prospective vs. retrospective In general populations the theoretical ceiling to predication is between 18-22% >This is higher in sicker, chronic populations Demographic models >Typically explain 2-5% of variance in utilization

19. 19 Metrics and Methods to Measure and Report Risk Adjustment Adding prior use increase the predicative ability of models to between 8 – 14% > Some antidotial evidence that adding Rx and Lab can increase predicative ability to the 18-22% range Adding health status changes the weights but does not increase the predicative ability of models

20. 20 Metrics and Methods to Measure and Report Traditional Health Care Performance Indicators Finances > Revenue and costs Clinical Activities Quality of Care > Includes access

21. 21 Metrics and Methods to Measure and Report Guidelines for Selecting Indicators The ability of systems to produce the data required to generate the indicator The perceived importance of the indicator to the development and assessment of the system The current utility of the indicator in organizations within the system The comparability of the indicator across systems The uniqueness of the applicability of the indicator

22. 22 Metrics and Methods to Measure and Report Indicators of System Performance Population Health > SF-30, SF-12, etc > Mental Health Index (such as the DIS) > Disease specific incidence and prevalence rates > Disease specific outcomes of care (SF-36, ADLs, IADLs)

23. 23 Metrics and Methods to Measure and Report Indicators of System Performance Quality of Care > Compliance with guidelines by type of provider > Incidence of Sentinel Events: - Unplanned re-visits - Avoidable hospitalisations - Hospital re-admission - Mortality

24. 24 Metrics and Methods to Measure and Report Indicators of System Performance Quality of Care (cont.): > Peer review > Disease specific mortality - Surgical - Infant mortality - Deaths due to cervical cancer > Frequency of preventable acute episodes for chronic conditions

25. 25 Metrics and Methods to Measure and Report Indicators of System Performance Quality of Care (cont.): > RX - Inappropriate poly pharmacy - ADRs Episode Prevention

26. 26

27. 27

28. 28

29. 29 Calculating Bed Days Works for any given time period (e.g. day, month to date, year to date) [A/ (B/365)] /(C/1000) > A = Gross bed days per unit of time > B = Days per unit of time > C = population size

30. 30 Knowledge Management Planning Cycle for Driving PCT Health Care System Performance Social/Health Indicators Evaluation Data & Analysis Implementation Useful Information Operations Planning Knowledge Management Strategy