1. Involving the Public in Priority Setting Involving the Public in Priority Setting A case study in NHS Dumfries and Galloway Verity Watson 1 Andrew Carnon 2 Mandy Ryan 1 Derek Cox 2 1 Health Economics Research Unit, University of Aberdeen 2 Directorate of Public Health and Strategic Planning, NHS Dumfries and Galloway

2. Structure 1.Introduction to priority setting 2.Overview of priority-setting process in NHS Dumfries & Galloway 3.How discrete choice experiments can be used in priority setting 4.Applying the discrete choice experiment in NHS Dumfries & Galloway 5.How successful was the priority-setting process?

3. How Do We Decide?

4. Introduction Options for health care are constrained by limited funding Choices imply priorities To make priorities explicit requires a priority- setting framework Options appraisal: –identified projects are scored based on pre-defined criteria –the relative importance of criteria are defined by weights Weights can lack transparency and accountability

5. Portsmouth Scorecard (Sandwell modification) Factor weightingVery low LowMidHighTop Evidence< 310203040 Benefit number< 310203040 Cost< 310203040 Acceptability< 310203040 National requirement< 310203040 Addressing inequality<35101520 Wider society benefit<35101520 Only alternative<35101520 Local feeling055710 (Austin et al, 2007)

6. Priority-Setting Principles in NHS Dumfries & Galloway Focus on Delivering for Health Transparency and rigour of process Public involvement Acute services CHP/Long term conditions Annual event Learning process

7. Priority-Setting Criteria Ten Criteria were chosen based on Delivering for Health 1.Location of care 2.Public consultation while developing project 3.Use of latest technology 4.Service availability 5.Patient involvement in own care 6.Management of care 7.Evidence of clinical effectiveness 8.Health gain 9.Risk avoidance 10.Priority area

8. NHS Dumfries & Galloway Programmes Acute Services Cancer CHD/Stroke/Diabetes Child Health Corporate Healthcare-Acquired Infection Learning Disabilities Local Health Partnership/Primary Care Long Term Conditions Mental Health Older People Public Health

9. Process in NHS Dumfries & Galloway Process in NHS Dumfries & Galloway Public involvement event 12 programme leads to submit bids Panel to take overview Bids scored on weighted criteria Ranked list of bids produced (Health Intelligence Unit) Corporate Management Team decisions

10. Public involvement event: Discrete Choice Experiment

11. Applying DCEs to Priority Setting To investigate the relative importance of criteria to public in Dumfries and Galloway Attributes are the 10 priority setting criteria Describe the criteria by a number of levels Define all possible combinations of attributes and levels Create choice sets Ask respondents to choose between different hypothetical health services for Dumfries and Galloway.

12. Attributes and Levels 1.Location of care at home at GP at Local Health Partnership at D&G Royal Infirmary outside D&G 2.Public involvement no consultation consultation at final stage consultation at some but not all stages consultation at all stages

13. Attributes and Levels 3.Use of Technology does not use latest technology uses latest technology uses cutting edge technology 4.Service availability office hours only office hours and outside office hours 5.Patient involvement in own care decision by health professional patient shares decision

14. Attributes and Levels 6.Management of care group of health professionals not working as a team an individual health professional group of health professionals working as team 7.Evidence of clinical effectiveness no evidence number of clinical studies at least one RCT at least three RCTs 8.Health Gain small gain to a small number large gain to a small number large gain to a large number

15. Attributes and Levels 9.Risk avoidance low risk to lower than low risk medium risk to low risk medium risk to lower than low risk high risk to medium risk high risk to low risk high risk to lower than low risk 10.Priority area none local priority national priority local and national priority

16. Experimental design 207,360 possible combinations (5 1 x6 1 x4 3 x3 3 x2 2 ) Used experimental design techniques and reduce to 64 profiles. These were paired with a mirror image (foldover method) Ensured orthogonality, minimum overlap and level balance Respondents were presented with the choice of two health services and asked to choose their most preferred.

17. Example choice

18. Sample and Setting A random sample of 100 members of public from Dumfries and Galloway Invited to attend a half day meeting Lunch and refreshments and travel expenses 68 respondents attended.

19. Sample and Setting

20. Age of Sample and Dumfries & Galloway Population

21. Rurality of Sample and Dumfries & Galloway Population

22. Deprivation of Sample and Dumfries & Galloway Population AdvantagedDisadvantaged

23. Analysis Responses are dichotomous, =1 for the option chosen and 0 for the option not chosen Analysis using a logit regression: Benefit = β i D xl All criteria are qualitative these are coded as dummy variables One level is omitted Coefficients are interpreted relative to the omitted level

24. Applying DCEs to Priority Setting

26. How were the Discrete Choice Experiment results used?

27. Example: Community/Voluntary Sector Health & Wellbeing Database CriterionLevelWeighted score LocationRegional0.34 Public involvementAll stages0.49 Use of technologyUp to date0.71 AvailabilityAll hours0.57 User involvement in treatment/service Has opportunity0.47 Management of careNot applicable- Evidence of effectivenessNot applicable- Health gainLarge gain to a large number0.97 Priority areaBoth local and national0.19 Weighted bid score =100 x Σ criterion weighted scores no of criteria scored 53.43

28. 0 10 20 30 40 50 60 1371013161922252831343740444648525458616467707376798285889194 Ranked Order Weighted Bid Score Acute ServicesCHP/LTCOther Bids Weighted Bid Scores

29. Programme Bid Scores

30. Top and Bottom Bids Top Scoring Bids 1.DGH capital developments 2.Additional consultant in elderly medicine 3.Expand anaesthetic services 4.Rapid access chest pain service 5.Community/voluntary sector health and wellbeing database 6.Extra IT training staff Bottom Scoring Bids 90.Set up neurology/genetic clinic 91.Increase paediatric clinics 92.Orthoptic vision screening 93.Expand school nursing 94.Appoint NHS Board business continuity manager 95.Provide extra secretarial input for paediatric consultants

31. Outputs of Priority-Setting Process Good spread of weighted bid scores Public involvement, transparency and accountability intrinsic to process Uncertain whether Delivering for Health principles fully reflected in weights (acute services and CHP/ long term conditions) Some anomalies (e.g. school nursing, IT training) Bid scores used as central aid to decision making, along with other factors (directives, risk, available finance)

32. How Successful was the Priority- Setting Process? Limitations Sample size Complex concepts for public Cost of exercise Decision making still difficult Benefits Planned approach to decision making Public response positive Transparency and defensibility Workable process

33. Organisational Priority-setting Decisions Relative Benefits Location Patient involvement Health gain, etc Difficulty Current state Availability of skills required Scale of change Urgency Risk if not done Quick wins possible Current versus avoided future costs Dependencies Funding available Fit with existing projects Basket of projects