1. Taken from presentations given at the 2013 National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Conference, London 9–10 October Great Quotes about Effectiveness Research

2. Contents – selected slides from: Dinner speech: Nifty shades of grey – that’s your HTA o John Gabbay, Emeritus Professor of Public Health, University of Southampton o Slides 3–9 Presentation: Reflections on the HTA Programme and where it is today o Sir Iain Chalmers, Health Services Researcher and Coordinator of the James Lind Initiative o Slides 10–11 Presentation: Comparative Effectiveness Research: Recent Developments in the USA o Sean Tunis, Founder, President & CEO, Center for Medical Technology Policy, USA o Slides 12–17

3. Most research seems to forget…that science serves human beings – social human beings who don’t fit into neat boxes, who are not the rational, linear thinking logical beings that many in the evidence ‐ based movement assume them to be; human beings who, on the contrary, are irrational and illogical. Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

4. …Successes are all about eschewing the false choices between black and white. It's about combining the two to find the right shades of grey. It’s about finding deft ways to resolve the “essential tensions 3 ” that we constantly live with. How else do clinicians and policy makers arrive at complex practical decisions in the ever different circumstances that beset their day ‐ to ‐ day practice? It's not about research or judgment, nor about RCTs or experience. It's always about both. 1.Thomas Kuhn first used this term in this context in his influential essay on divergent and convergent thinking in science and education. Kuhn T S (1977) The Essential Tension Chicago University Press. Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

5. We argue between two extremes, often taking up entrenched positions, when deep down we know that they are forced, false choices: Primary research or secondary research? RCTs or observational studies? Quantitative or qualitative? Carrots or sticks? Black or White? The answers may well not be one or the other but usually “both”. Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

6. …with our ageing population how clinically useful are the results of non ‐ pragmatic trials restricted to patients under 65 years of age with no comorbidity, unless clinicians mostly add gallons of clinical judgment when making decisions based on them? Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

7. We still find ourselves debating whether to do: Ethnographic work on the nature of a complex intervention to see if it works o No! If you don't measure it, it's not science!, or Do trials o No! If you can measure it, it’s probably not “it”! Why not much more often consciously and systematically go for both? After all, both are necessary and neither is sufficient. Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

8. Trials, even perfectly executed pragmatic ones, are not and never could be the whole story. Pursuing the development of other methods, even of rigorous critically appraising evidence other than research evidence, may pay dividends. Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

9. Given that people’s views of scientific findings are so coloured by their “tribal” allegiances … [it is astute to include] a wide spectrum of different interest groups throughout the entire process, so that their views can be absorbed …a suitably counterbalanced way…a long list of “P”s: o Patients, Physicians, Professionals, Providers, Purchasers/ Payers, Public health, Policy makers, Politicians, Press, Public, and Producers of the technologies, e.g. Pharma, and also the Primary research. It is by [being] inclusive, so that differing values and interests are at least debated and negotiated if not neutralised, that ensures research actually fits its context Nifty shades of grey – that’s your HTA John Gabbay, Emeritus Professor of Public Health, University of Southampton

10. Reflections on the HTA Programme and where it is today Sir Iain Chalmers, Health Services Researcher and Coordinator of the James Lind Initiative “I need permission to give a drug to half of my patients, but not to give it to them all.” Richard Smithells

11. Reflections on the HTA Programme and where it is today Sir Iain Chalmers, Health Services Researcher and Coordinator of the James Lind Initiative “The clinician who is convinced that a certain treatment works will almost never find an ethicist in his path, whereas his colleague who wonders and doubts and wants to learn will stumble over piles of them.” Lancet Editorial 1990

12. Comparative Effectiveness Research: Recent Developments in the USA Sean Tunis, Founder, President & CEO, Center for Medical Technology Policy, USA The tipping point for comparative effectiveness research in the USA came in 207: “Better information about the costs and benefits of different treatment options, combined with new incentive structures reflecting the information….is essential to putting the country on a sounder long- term fiscal path.” Peter Orszag, Congressional Budget Office Senate testimony, June 2007

13. Comparative Effectiveness Research: Recent Developments in the USA Sean Tunis, Founder, President & CEO, Center for Medical Technology Policy, USA Long-term fiscal gap and healthcare costs

14. Comparative Effectiveness Research: Recent Developments in the USA Sean Tunis, Founder, President & CEO, Center for Medical Technology Policy, USA Comparative Effectiveness Research (CER) hypothesis: o Gaps in evidence will be reduced with greater engagement of end users (decision makers = patients, clinicians, payers) in –Identifying most important uncertainties –Developing study protocols / methods –Implementing studies –Applying evidence in context of payment reforms that reward efficiency

15. Comparative Effectiveness Research: Recent Developments in the USA Sean Tunis, Founder, President & CEO, Center for Medical Technology Policy, USA