1. Professor David Parkin King’s College London
Using Patient Reported Outcome Measures to monitor health care performance
Professor David Parkin
King’s College London

2. The NHS PROMS initiative
In April 2009, the Department of Health required routine measurement of patient-reported health outcomes for all NHS patients in England, before and after four surgical procedures: hip and knee replacements and hernia and varicose vein repairs.
All NHS providers currently collect these data; around patients are invited each year to complete questionnaires; good response rates, for example 80% for hip replacement.
Those four procedures cost the NHS in England around £800 million per year. The estimated annual cost of the current PROMs programme is less than 0.5 per cent of that.
Currently pilot work on a range of chronic conditions; aim to introduce across a wide range of NHS services.

3. The NHS PROMS initiative
For all of the procedures, the EQ-5D is used.
For hip and knee replacements and varicose vein repairs, a condition specific instrument is also included – Oxford Hip and Knee Scores; Aberdeen Varicose Vein Score

4. Health gain following hip replacement by hospital: England (2012/13)

5. Case-mix adjusted average health gain: Primary knee replacement, English hospitals, April 2012 – March 2013

6. Outcomes of elective procedures in NHS South East Coast
1.000
0.889
0.885
0.900
0.805
0.789
0.786
0.800
0.729
0.700
0.600
Hip replacement
Knee replacement
EQ-5D Index
0.500
0.444
Varicose veins
0.420
Groin Hernia
0.391
0.396
0.400
0.300
0.285
0.291
0.200
0.100
0.105
0.100
0.081
0.079
0.000
Pre-op
Post-op
Gain
Case mix adjusted gain

7. Hip Replacement Pre-op Post-op Gain Case mix adjusted gain
SOUTH EAST COAST
0.391
0.786
0.396
0.420
NORTH EAST
0.335
0.747
0.413
0.416
EAST OF ENGLAND
0.340
0.764
0.423
0.414
SOUTH CENTRAL
0.783
0.387
0.409
England
0.354
0.761
0.407
SOUTH WEST
0.380
0.772
NORTH WEST
0.427
0.405
LONDON
0.336
0.743
0.402
YORKSHIRE AND THE HUMBER
0.332
0.752
EAST MIDLANDS
0.744
0.412
0.399
WEST MIDLANDS
0.348
0.746
0.398

8. Hip replacement - SEC providers in context

9. Hip replacement - all providers in England

10. Hip replacement - 95% confidence intervals

11. Creating a health index using profiles and weights
One way of creating a health status index for a person is to use a weighted ‘health state profile’, which is purely descriptive.
The profile is generated using data on the person from a questionnaire which assigns ‘levels’ to ‘dimensions’.
These levels within dimensions are summed to produce a single number index
Most condition specific indexes are ‘unweighted’, treating levels as ranks and summing over dimensions.
Most generic indexes weight each level within each dimension, with weights taken from a separate ‘valuation’ survey

12. Paretian categories for EQ-5D change
Four possible outcomes from an intervention: 1. The health profiles are the same: there has been no change in health state. 2. The second profile is better than the first: there has been an unequivocal improvement in health. 3. The second profile is worse than the first: there has been an unequivocal worsening in health. 4. The first and second health profiles are non-comparable: there has been a change in health but we cannot say if it is an improvement or worsening.

13. Paretian classification of health change using EQ-5D profile data before and after four elective procedures

14. Health Profile Grid: the distribution of individual changes in health
Paretian categories: partial ordering
Impose a full ordering, so all 243 states are ranked from best to worst.
Facilitates a simple diagrammatic display of the individual changes in health:
EQ-5D profile before surgery on the vertical axis
EQ-5D profile after surgery on the horizontal
The 45˚ line represents no change in health

15. Changes in health state for hip operations (PROMs pilot data)

16. Changes in health state for cataract operations (PROMs pilot data)

17. Case mix adjustment – Step 1 (EQ-5D Hip replacement example)
Estimate explanatory model for post-operative health state using these categories of variables (those used in the example in parentheses) :
Patient demographics (Sex)
Other patient characteristics (Did not consider themselves disabled; had assistance in completing the Q2 questionnaire)
Clinical risk variables derived from HES (Total hip replacement or hybrid prosthetic hip revisions; COPD and rheumatoid arthritis)
Clinical risk variables from the patient questionnaires (Reported circulatory, arthritis or anxiety and depression; baseline index)
Other risk factors from the patient questionnaires (General health question)
Local area related variables (Index of Multiple Deprivation)
Provider related variables (Post-operative length of stay; time from procedure to completion of Q2)

18. Case mix adjustment – Step 2
Calculate for for each Provider predicted post-operative scores.
The values of Local Area and Provider related variables for individual providers are not used in this; replaced by an addition to the constant, the coefficient multiplied by the average value for the sample.
This gives a comparison with a hypothetical provider that has the same case mix, rather than with other real providers. Each provider’s outcomes are therefore adjusted to compare to a standardised case mix.
This mix is the average level of the case-mix variables over all providers, which by definition also generates the all-providers average PROMs score.
The change in health state is therefore calculated as the adjusted post-operative score minus the average pre-operative score over all providers

19. X X X X Provider A actual Q2 Provider A expected Q2 Q2 Regression line
Q2c
Q1=Q2 line X
Q2d
ΔQ’ X
Q2a X
Q2b
Provider A actual Q2 if it had the all providers average Q1 ΔQ
Actual all providers average Q2 = expected all providers average Q2 Q1
Provider A average Q1a
All providers average Q1

20. Mean 0.405
Std. dev
Min 0.266
Max 0.544
Range 0.278
Mean 0.405
Std. dev
Min 0.314
Max 0.488
Range 0.174

21. Adjusted and unadjusted EQ-5D index scores for primary hip replacement: English hospitals, 2012/2013: ordered on both adjusted and unadjusted results

22. Adjusted and unadjusted EQ-5D index scores for primary hip replacement: English hospitals, 2012/2013: ordered on unadjusted results only