1. Copyright 2011 Right Care IDEAL A rational approach to evaluation of complex treatments Peter McCulloch Chair, IDEAL Collaboration March 2015 Online Learning Series Right Care for Populations

2. Developing Evidence for Therapy PHARMA PARADIGM Theory Lab demonstration (animal studies) First-in-man study Toxicity study (Phase I) Efficacy Study (Phase II) RCT (Phase III) Post-Marketing Surveillance (Phase IV) SURGICAL PARADIGM Idea (animal/bench trial) First-in-man (CS) Outcome experience (CS) Confirmation/dispute from others (more CS) RCT??

3. SPECIAL PROBLEMS OF SURGICAL TRIALS 1.Early “tinkering” 2.Defining the surgical intervention 3.Accommodating surgical learning curves 4.Quality Control 5.Developing standard outcome measures 6.Strong preferences

4. IDEAL: What and Why? AIM Improve the methodological quality of research in surgery and interventional therapies STEP 1: FRAMEWORK Describe the natural history of surgical innovation STEP 2: RECOMMENDATION Define appropriate studies for each stage in the innovation pathway

5. Stage 1 IDEA Stage 2a DEVELOPMENT Stage 2b EXPLORATION Stage 3 ASSESSMENT Stage 4 LONG TERM MONITORING Initial report Innovation may be planned, accidental or forced Focus on explanation and description “Tinkering” (rapid iterative modification of technique and indications) Small experience from one centre Focus on technical details and feasibility Technique now more stable Replication by others Focus on adverse effects and potential benefits Learning curves important Definition and quality parameters developed Gaining wide acceptance Considered as possible replacement for current treatment Comparison against current best practice (RCT if possible) Monitoring late and rare problems, changes in use & quality of surgical performance

6. Identifying the IDEAL Stage Quick literature search Review article TYPES, NUMBER, SOURCES and PATIENT NUMBERS Use the Framework to identify the Stage Leads on to RECOMMENDATIONS for evaluation

7. Stage 1: IDEA RECOMMENDATIONS Professional Innovation Database Compulsory reporting of all new innovations on a Professional Innovation Database Confidential entry allowed to encourage reporting of failed innovations (CHRP system) “Light touch” local unified regulation

8. Stage 2a: Development RECOMMENDATIONS Prospective Development Studies Prospective Development Studies: - Detailed description of selection criteria -Detailed technical description -Prospective account of ALL cases consecutively - Clear STANDARDISED definitions of outcomes reported -Description of ALL modifications and when made in series Registration of PROTOCOL before study starts

9. PDS Example: Development of Robotic Oesophagectomy Hospital Stay (days)

10. Stage 2b Exploration RECOMMENDATIONS Prospective Exploration Study (collaborative uncontrolled prospective study ) To evaluate technique prospectively and co-operatively To develop a consensus over definition of the procedure, quality standards To accumulate data for power calculations To evaluate and monitor learning curves To evaluate preferences and values amongst patients and clinicians To achieve consensus on the trial question To develop a multi-centre randomised trial

11. PES Example: Feasibility Study for RCT of radical gastrectomy 8 North Italian surgical units Training scheme Prospective Cohort Review of short term outcomes 5 Eligible groups enter RCT Extended lymph node dissection for gastric cancer: results of a prospective, multi-centre analysis of morbidity and mortality in 118 consecutive cases. Degiuli M, Sasako M, Ponzetto A, Allone T, Soldati T, Calgaro M, Balcet F, Bussone R, Olivieri F, Scaglione D, Danese F, Morino M, Calderini P, Capussotti L, Fronda G, Garavoglia M, Locatelli L, Dellepiane M, Rossini FP, Calvo F. Eur J Surg Oncol. 1997 Aug;23(4):310-4. Morbidity and mortality in the Italian Gastric Cancer Study Group randomized clinical trial of D1 versus D2 resection for gastric cancer. Degiuli M, Sasako M, Ponti A; Italian Gastric Cancer Study Group. Br J Surg. 2010 May;97(5):643-9. doi: 10.1002/bjs.6936.

12. Surgical RCT Stage 3: Assessment RECOMMENDATIONS Surgical RCT Or Alternative… Consider blinding Standardise Terms Use Stage 2b data to decide; Power calculations Definition of intervention Quality measures Learning curve eligibility Deal with preferences Expertise-based trials Qualitative research Third party randomisation Decision support aids Cohort/RCT mixtures Feasibility trials(?) Stepped-wedge design Controlled interrupted time series

13. Stage 4 Long term study RECOMMENDATIONS Prospective Registries Prospective Registries Best designed by clinical community (with help) Best curated by impartial body (not manufacturer) SPC used for quality control (Shewart charts, CUSUM, VLAD)

14. IDEAL An integrated evaluation pathway REGISTRATION OF 1 st in MAN PROSPECTIVE DEVELOPMENT STUDY PROSPECTIVE EXPLORATION STUDY RCT REGISTRY

15. Uses and Implications For designing and publishing studies of invasive therapeutic techniques For defining the evidence required for licensing of therapeutic devices & implants For deciding what evidence is needed to justify commissioning of procedures and devices

16. Get involved Follow us on Twitter @IDEALCollab@IDEALCollab Link to our website from your institution’s website Send us text about your work or IDEAL related topics so we can upload as news items on the website www.ideal-collaboration.net Become a member and subscribe to our newsletter Collaborate on piloting & evaluating IDEAL study designs Contact: allison.hirst@nds.ox.ac.uk IDEAL Project Manager Peter.mcculloch@nds.ox.ac.uk IDEAL Chair

17. 17 Online Learning Series Right Care for Populations Follow Right Care online Subscribe to get a weekly digest of our blog alerts in your inbox, Receive occasional eBulletins Follow us on Twitter @qipprightcare Find the full series at: www.rightcare.nhs.uk/resourcecentre Find more IDEAL resources at: www.rightcare.nhs.uk/resourcecentre