Measuring quality processes and outcomes Rod O'Connor PhD Rod O'Connor & Associates P/L Adjunct Senior Lecturer in Health Outcomes Measurement, SHSM, UNSW 59 Gipps St. Birchgrove NSW Ph: Mob:

Measurement of outcome quality Origins:  Economic - Demand to assess rival health programs in a context of increasing pressure on resources  Medical effects - move to comprehensively evaluate clinical therapies  Recognition that patients assess their health differently to their doctors Measuring Quality Processes and Outcomes

U.S. President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry - ‘Quality First: Better Health Care for All Americans’ Role of quality measures in internal quality improvement initiatives - u ‘provide the tools that allow health care providers and organizations to undertake focused improvement programs to improve the quality of care they furnish, whereby quality is measured both before and after an intervention’ Measuring Quality Processes and Outcomes

Need to capture multiple dimensions of quality u the processes associated with the delivery of care u the outcomes obtained Measuring Quality Processes and Outcomes

Treatment (process) and outcome indicators are both required:  treatment indicators to reveal the source of outcome variation, and because case mix variation can make outcomes hard to interpret  outcome indicators to reveal the state of health of patients (and ex-patients), and to identify unexpected effects and effects of most concern to the patient - mortality, morbidity, functional status, HQOL, and ‘satisfaction’ all relevant. Measuring Quality Processes and Outcomes

Desirable to have core data sets, evaluated against criteria (U.S. report) u Scientific soundness (i.e., reliable, valid, adjusted) u Importance of the quality concern; u Relevance to various users; u Potential to foster improved health status or well-being u Evidence basis u Interpretability, and Actionability ( the degree to which steps can be taken to address the concern); u Feasibility; and Ease/ Cost-effectiveness of measurement. Measuring Quality Processes and Outcomes

But essential to encourage innovation (U.S. report) u Identification of core sets of quality measures should not lock in approaches to quality measurement and reporting. u Innovation can be fostered by … placing limits on the breadth of standardization efforts (e.g., number of measures) so compliance with the core reporting sets does not consume resources better devoted to innovation or internal quality improvement Measuring Quality Processes and Outcomes

Practical measurement Issues 1: Measures need to be valid u Important that instruments measure what they are intended to u Increasing focus on measuring care from the patient’s perspective Measuring Quality Processes and Outcomes

The cost of poor measures The cost of poor measures is wasted and potentially misleading information, potentially leading to: u failure to improve services u discrimination against good treatments u development of poor quality services Measuring Quality Processes and Outcomes

Need to specify the measure, its purpose, and the action that flows from variation in the measure u What is the decision to be made? u Which variable needs to be estimated to guide the decision? u How can the variable be most simply & directly measured? u What degree of change is needed to provoke a response, and what should the response be? Measuring Quality Processes and Outcomes

Practical measurement Issues 2: Indicators need explicit definition, quantification, and standardisation u Each indicator, and its purpose, need to be clearly defined, as well as the conditions under which it is to be measured. u quality measures need standard operational definitions, eg. caesarean section rates need to specify the age range of women to be included in the measure, the timeframe etc. Measuring Quality Processes and Outcomes

Indicators require: a. operationalised definitions eg. 'some reduction in dexterity', given the example of 'difficulty smoothly placing key in lock'. b. explicit quantification of phrases by removing terms such as 'severely' and 'moderately’ - thus 'seriously disruptive' became 'disrupts work for at least 60 minutes per day'. c. domains clarified & emphasised - eg. ‘communication difficulties’ refers to sensory impairment, not cognitive impairment or language knowledge. Measuring Quality Processes and Outcomes

Practical measurement Issues 3: When assessing outcomes, may need to make adjustments for risk (particularly with one-off measures, eg. mortality) u Outcome indicators can employ mortality data, clinical outcomes, HQOL and functional status measures (disease-specific or generic). But outcomes result from a complex mix of factors that relate to intrinsic patient characteristics as well as quality of care. Measuring Quality Processes and Outcomes

The problem of attribution Adjusting for differences in baseline risk can be extremely difficult  persons with complex illnesses, multiple coexisting diseases, or other significant risk factors are more likely to do poorly, even with the best care, than healthy individuals (Iezzoni, 1997).  there is often a long delay between care given and an outcome occurring, leading to uncertainty as to which episode of care (or which carer) was responsible.  outcomes of interest may be infrequent (the "small n" problem) Measuring Quality Processes and Outcomes

Practical measurement Issues 4: Patient-centred outcomes are important, as well as clinical outcomes u Most outcomes research still focuses on mortality and morbidity rates (did test-group patients live longer, show significant improvements in physiological measures of health status, were there untoward side-effects or complications). Relatively few research studies have tended to assess outcomes in terms of HQOL(health-related quality of life), disability, patient satisfaction. Measuring Quality Processes and Outcomes

Patient-centred measurement, such as HQOL, can potentially change treatment practices by identifying relatively neglected areas of major concern to patients Measuring Quality Processes and Outcomes

u Patients assess their health differently to their doctors, both according to degree of improvement and the areas of greatest concern. u Doctors tend to stress physical much more than psycho ‑ social aspects of health u The separation between doctor and patient judgements tends to increase as the doctor becomes more senior. Measuring Quality Processes and Outcomes

Patient-focussed outcomes include: u the patient's report of the degree to which their overall health has improved u the extent to which the functional and quality- of-life areas of greatest concern to patients have been remedied u patient perceptions of the health care experience (beware of reports on 'satisfaction'). Similarly, ‘appropriateness of care’ is ultimately assessed according to the extent it delivers outcomes valued by the patient) Measuring Quality Processes and Outcomes

Psychometric measures can develop better services from the patient’s perspective Illness-specific measures of health-related quality of life (based on self-report) can be systematically decomposed to reveal the areas of key importance to patients, which typically differ from the judgement of doctors. Thus HQOL measures can play a diagnostic as well as an outcome-measurement role. Measuring Quality Processes and Outcomes

Caution: When using generic HQOL measures, recognise that they may be insensitive and misleading - test them rigorously Eg. Sickness Impact Profile (SIP) u For all patients, self-assessed dysfunction & global SIP, r =0.52; for speech pathology only, r =-0.01 Measuring Quality Processes and Outcomes

The Present Concerns regarding the validity and application of existing indicators, concerns regarding risk adjustment, relatively little attention to validity/reliability. HQOL and client satisfaction tests considered soft and of doubtful merit. Where HQOL used, usually reliance on omnibus measures. Measuring Quality Processes and Outcomes

The Future The disciplines of epidemiology, psychometrics, and health economics will become more integrated. Clinical measures and cost/ efficiency, HQOL/ functionality, and client satisfaction measures will approach equal status. HQOL and other psychometric tests will play a major diagnostic role, to identify issues and areas most important to the patient, and target services/treatments to them. Measuring Quality Processes and Outcomes