Professor David Parkin King’s College London Using Patient Reported Outcome Measures to monitor health care performance Professor David Parkin King’s College London

The NHS PROMS initiative In April 2009, the Department of Health required routine measurement of patient-reported health outcomes for all NHS patients in England, before and after four surgical procedures: hip and knee replacements and hernia and varicose vein repairs. All NHS providers currently collect these data; around 250 000 patients are invited each year to complete questionnaires; good response rates, for example 80% for hip replacement. Those four procedures cost the NHS in England around £800 million per year. The estimated annual cost of the current PROMs programme is less than 0.5 per cent of that. Currently pilot work on a range of chronic conditions; aim to introduce across a wide range of NHS services.

The NHS PROMS initiative For all of the procedures, the EQ-5D is used. For hip and knee replacements and varicose vein repairs, a condition specific instrument is also included – Oxford Hip and Knee Scores; Aberdeen Varicose Vein Score

Health gain following hip replacement by hospital: England (2012/13)

Case-mix adjusted average health gain: Primary knee replacement, English hospitals, April 2012 – March 2013

Outcomes of elective procedures in NHS South East Coast 1.000 0.889 0.885 0.900 0.805 0.789 0.786 0.800 0.729 0.700 0.600 Hip replacement Knee replacement EQ-5D Index 0.500 0.444 Varicose veins 0.420 Groin Hernia 0.391 0.396 0.400 0.300 0.285 0.291 0.200 0.100 0.105 0.100 0.081 0.079 0.000 Pre-op Post-op Gain Case mix adjusted gain

Hip Replacement Pre-op Post-op Gain Case mix adjusted gain SOUTH EAST COAST 0.391 0.786 0.396 0.420 NORTH EAST 0.335 0.747 0.413 0.416 EAST OF ENGLAND 0.340 0.764 0.423 0.414 SOUTH CENTRAL 0.783 0.387 0.409 England 0.354 0.761 0.407 SOUTH WEST 0.380 0.772 NORTH WEST 0.427 0.405 LONDON 0.336 0.743 0.402 YORKSHIRE AND THE HUMBER 0.332 0.752 EAST MIDLANDS 0.744 0.412 0.399 WEST MIDLANDS 0.348 0.746 0.398

Hip replacement - SEC providers in context

Hip replacement - all providers in England

Hip replacement - 95% confidence intervals

Creating a health index using profiles and weights One way of creating a health status index for a person is to use a weighted ‘health state profile’, which is purely descriptive. The profile is generated using data on the person from a questionnaire which assigns ‘levels’ to ‘dimensions’. These levels within dimensions are summed to produce a single number index Most condition specific indexes are ‘unweighted’, treating levels as ranks and summing over dimensions. Most generic indexes weight each level within each dimension, with weights taken from a separate ‘valuation’ survey

Paretian categories for EQ-5D change Four possible outcomes from an intervention: 1. The health profiles are the same: there has been no change in health state. 2. The second profile is better than the first: there has been an unequivocal improvement in health. 3. The second profile is worse than the first: there has been an unequivocal worsening in health. 4. The first and second health profiles are non-comparable: there has been a change in health but we cannot say if it is an improvement or worsening.

Paretian classification of health change using EQ-5D profile data before and after four elective procedures

Health Profile Grid: the distribution of individual changes in health Paretian categories: partial ordering Impose a full ordering, so all 243 states are ranked from best to worst. Facilitates a simple diagrammatic display of the individual changes in health: EQ-5D profile before surgery on the vertical axis EQ-5D profile after surgery on the horizontal The 45˚ line represents no change in health

Changes in health state for hip operations (PROMs pilot data)

Changes in health state for cataract operations (PROMs pilot data)

Case mix adjustment – Step 1 (EQ-5D Hip replacement example) Estimate explanatory model for post-operative health state using these categories of variables (those used in the example in parentheses) : Patient demographics (Sex) Other patient characteristics (Did not consider themselves disabled; had assistance in completing the Q2 questionnaire) Clinical risk variables derived from HES (Total hip replacement or hybrid prosthetic hip revisions; COPD and rheumatoid arthritis) Clinical risk variables from the patient questionnaires (Reported circulatory, arthritis or anxiety and depression; baseline index) Other risk factors from the patient questionnaires (General health question) Local area related variables (Index of Multiple Deprivation) Provider related variables (Post-operative length of stay; time from procedure to completion of Q2)

Case mix adjustment – Step 2 Calculate for for each Provider predicted post-operative scores. The values of Local Area and Provider related variables for individual providers are not used in this; replaced by an addition to the constant, the coefficient multiplied by the average value for the sample. This gives a comparison with a hypothetical provider that has the same case mix, rather than with other real providers. Each provider’s outcomes are therefore adjusted to compare to a standardised case mix. This mix is the average level of the case-mix variables over all providers, which by definition also generates the all-providers average PROMs score. The change in health state is therefore calculated as the adjusted post-operative score minus the average pre-operative score over all providers

X X X X Provider A actual Q2 Provider A expected Q2 Q2 Regression line Q2c Q1=Q2 line X Q2d ΔQ’ X Q2a X Q2b Provider A actual Q2 if it had the all providers average Q1 ΔQ Actual all providers average Q2 = expected all providers average Q2 Q1 Provider A average Q1a All providers average Q1

Mean 0.405 Std. dev. 0.045 Min 0.266 Max 0.544 Range 0.278 Mean 0.405 Std. dev. 0.034 Min 0.314 Max 0.488 Range 0.174

Adjusted and unadjusted EQ-5D index scores for primary hip replacement: English hospitals, 2012/2013: ordered on both adjusted and unadjusted results

Adjusted and unadjusted EQ-5D index scores for primary hip replacement: English hospitals, 2012/2013: ordered on unadjusted results only