1. Technology Appraisal of Medical Devices at NICE – Methods and Practice Mark Sculpher Professor of Health Economics Centre for Health Economics University of York, UK

2. Outline Policy context of NICE NICE methods Are devices different? The role of randomised trials When do we have sufficient evidence? Impact of NICE guidance

3. Background Brief overview of NICE NICEs decisions on devices NICEs methods: requirement for decision making Are devices different? The impact of NICE guidance

4. The National Institute for Health and Clinical Excellence (NICE) Following election of Labour government 1997 Prolonged controversy about post code prescribing in the UK National Health Service Wish to de-politicize decisions about which technologies to cover in NHS Desire to use best available methods to address difficult questions Focus on drugs but devices also included

5. The NICE process SelectionAssessmentAppraisal Specific technologies Lacking in transparency Subject to some criteria Independent group Review plus model Good methods supported Assess company submissions 6 months or more Companies can also provide unpublished data Multi-disciplinary committees Take information from range of sources Range of decisions possible

6. NICE decisions overall Source: Raftery, BMJ 2006.

7. NICE and medical devices (1) Source: Raftery, BMJ 2006.

8. NICE and medical devices (2) Source: Raftery, BMJ 2006.

9. NICE and methods 2004 guidance more prescriptive than most international guidelines Based on a clear view about the use of economic evaluation for decision making Included the concept of the Reference Case National Institute for Clinical Excellence (NICE). Guide to the Methods of Technology Appraisal. London: NICE, 2004.

10. The requirements of economic evaluation for NICE-decision making Objective function Generic measures of health; QALYs Decision problem Clarity about population; full specification of options Appropriate time horizon Evidence base Context Time over which options might differ Inclusion of all relevant evidence Relevant to specific decision maker(s) Uncertainty Quantify decision uncertainty; feed in research prioritisation

11. NICEs preferred methodology – the Reference Case Source: National Institute for Clinical Excellence (NICE). Guide to the Methods of Technology Appraisal. London: NICE, 2004.

12. What is the appropriate framework for economic evaluation? Evidence synthesis Decision analysis Systematic review Meta-analysis Mixed treatment comparisons Differing endpoints and follow-up Patient-level and summary data Structure reflecting disease Incorporation of evidence on range of parameters Facilitates extrapolation and separation of baseline and treatment effects Probabilistic methods

13. Are devices different? Decision problem Need to include all relevant alternatives to the technology of interest –May include pharmaceuticals –May include sequences and other strategies (e.g. diagnostic) Need to define relevant populations and sub- populations May differ between jurisdictions No clear differences between devices and pharmaceuticals

14. Are devices different? Evidence base Less likely to need trials for regulatory purposes Does not mean should not be used for reimbursement Typical regulatory trials have limitations for economic evaluation The evolution of devices over time –Not unique to devices –Has implications for evidence gathering –Need larger longitudinal studies, sub-groups on device types –Comparators may also be changing over time

15. Limitations of trials as a vehicle for decision making Trial limitations Inappropriate or partial comparisons More than one trial Partial measurement Unrepresentative practice Intermediate outcomes Limited follow-up No trials NICE Examples Temozolomide (recurrent malignant glioma) Drugs for Alzheimers Riluzole (resource use) Glycoproteins Beta interferon (MS) Implantable cardioverter defibrillators Liquid-based cytology

16. Distinguishing types of parameters in models Type of parameter Baseline events risks Relative treatment effects Resource use/costs Quality of life/utilities Sources RCT control groups Observational RCTs preferred RCTs or observational Synthesis issues Often long-term Often by sub-group Usually limited head- to-head data Formal synthesis rare Frequent need to map Formal synthesis rare Cost-effectiveness models invariably draw evidence from multiple sources

17. Making trials more naturalistic The design continuum Comparators Placebo controlled All relevant comparators Measurement Few efficacy and safety endpoints Resource use, QoL Follow-up Shortest acceptable for registration Long-term follow-up Patients Tightly defined Reflective of full range of likely patients Practice Highly protocolised Reflective of routine practice

18. When is there sufficient evidence to reimburse? Decision uncertainty Implications of getting it wrong Value of perfect information What is the probability of the wrong decision? Joint effect of uncertainty in all inputs What are the implications of a wrong decision in terms of resources and health? = Sets an upper bound on the value of further research Can be calculated overall and for individual parameters Calculated per patient and across a population of patients X Sufficient evidence exists if it is not cost-effective to undertake further research

19. Evidence on impact of NICE decision on the NHS

20. Evidence on Orlistat for obesity Source: Sheldon et al. BMJ 2004;329:999.

21. Evidence on ICDs for arrhythmias Source: Sheldon et al. BMJ 2004;329:999.

22. What influences uptake? Source: Sheldon et al. BMJ 2004;329:999.

23. Conclusions NICE part of an international trend towards greater use of economics in decision making NICE is prescriptive about methods Few differences between drugs and devices which affect appropriate methods Impact of NICE guidance has been variable