1. COSBBI 2013-07-10 DBMI Roger Day Personalized medicine Clinical trials p. 1

2. Individuals and the group: the challenge of “personalized medicine” p. 2 Why do we need statistics in medicine? Because people are individuals (not because they’re all alike). It’s all about VARIABILITY On the other hand… “I take comfort in thinking of myself as a statistic” ---Nan Laird, Former Chair, Dep’t of Biostats, Harvard School of Public Health … What a medicine does to me teaches something about what it does to you.

3. The Lump/Split Dilemma A new treatment is given to 100 patients. Of them, only 8 respond. But there is a subgroup of 5 in which 3 patients respond, yielding a response rate of 60%! Should the treatment be recommended for people in the subgroup? (“Personalized”?) p. 3

4. Lump? Split? Dr. Lump: “Of course hair color has nothing to do with it. Response rate = 8/100. Don’t treat the Dark Hair people!” Dr. Split: “The latest research says personalize. Response rate = 3/5=60%. Treat the Dark Hair people!” p. 4

5. Some Bayesian analyses p. 5

6. Dr. I. DontKnow: Let the data tell us˜ p. 6

7. p. 7 X =Observation (Dr. Split) Green = prior belief = prior mean Red = posterior belief (prior + data) M =posterior mean

8. Goldilocks and the three investigators Dr. Lump: low variance, high bias Dr. Split: high variance, low bias Dr. I.Dontknow: JUST…right! –Empirical Bayes, hierarchical model The challenge for the future of medicine: Let DATA + PRIOR UNDERSTANDING dictate how much to “personalize”. p. 8

9. p. 9

10. CLINICAL TRIALS! p. 10

11. p. 11 Drug development process (idealized) Molecular In vitro In vivo Epidemiologic “File cabinet” { IDEA Phase IOBJ: “Safety” ENDPT: Toxicity ) Phase IIOBJ: “Efficacy” ENDPT: Clinical response Phase IIIOBJ: “Effectiveness” ENDPT: Survival Phase IVOBJ: “Outcomes” ENDPT: Pain, cost, …

12. p. 12 Phase I Clinical Trials: Objectives OBJECTIVES: Identification of toxicities to watch out for. Determination of a “Recommended Phase II Dose”. DEFINITION: Maximum tolerated dose (MTD): “The highest level of a dose that can be tolerated” (“Tolerated”= an “acceptable” risk of toxicity) DEFINITION: Dose-limiting toxicity (DLT): (1) “an adverse event that is counted against dose escalation” (2) “a type of adverse event associated with the drug being tested”

13. p. 13 Phase I Clinical Trials: Endpoints Severity grades: 1Mild 2Moderate DLT3Severe DLT 4 Life-threatening DLT 5Fatal The U.S. National Cancer Institute: Common Toxicity Criteria 1982; CTC Version 2.0 1998 ; Common Terminology Criteria for Adverse Events v3.0 (CTCAE) (an informatics data conversion nightmare)

14. p. 14 Phase I: Standard Design (3+3) 3 patients per dose tier If #DLT =... 0/3, then Escalate the dose 1/3, then add 3 pts if 1/6, then Escalate if any more, then Stop 2/3,then Stop (many better designs, but still the most popular)

15. p. 15 What’s a “good” toxicity rate? proportion of patients who will get adverse events 0 0.1 0.2 0.3 0.4 0.5 “too toxic, no matter what!” “just right” “not toxic enough- probably won’t work.”

16. Development of a Phase I Dose from a Dose-Ranging Study p. 16

17. Let’s play Phase I Trial! You are the Patient. p. 17

18. p. 18

19. p. 19

20. p. 20 How much information do we really have about the “maximum tolerated dose” If 0 out of 3 DLT’s, –estimated risk of DLT is “zero” –95% confidence interval is 0% to 63%. If 1 out of 6 DLT’s, –estimated risk of DLT is 17% –95% confidence interval is 4% to 64%. ---- not much information!!!

21. p. 21 Main purpose: Determine whether there is sufficient evidence of efficacy Secondary: Determine (or confirm) safety with greater confidence. (Is the “MTD” or “RP2D” really “tolerable”?) Usually just one regimen. Often just one drug or other treatment at a time. Phase II: Objectives

22. p. 22 Efficacy vs Effectiveness: Textbook Definitions Efficacy –“true biological effect of a treatment” Effectiveness –“the effect of a treatment when widely used in practice”

23. p. 23 If early results say the treatment is not very good, isn’t it UNETHICAL to continue to accrue patients? ETHICS |||||||| DESIGN ||||||| BIOSTATISTICS Solution: An early stopping rule a special case of ADAPTIVE DESIGN. Phase II: Ethical Issue

24. p. 24 Some Phase II study designs “Simon design” - Early stopping for poor response “Bryant-Day design - Early stopping for poor response or excess toxicity

25. p. 25 This might be a “Type II error”. This might be a “Type I error”. Simon two-stage design, Control Clin Trials. 1989

26. p. 26 Study Design Jargon “Type I Error” (or “alpha”) The probability that you say “whoopee” when you shouldn’t. “Type II Error” (or “beta”) The probability that you say “poopie” … when you shouldn’t. “Statistical Power” The probability that you say “whoopee”... when you should. So.... Power = 1 – beta = 1 – Type II Error

27. p. 27 Response rates: What’s good enough? What’s too bad? proportion of patients who will respond 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 we love the new treatment! indifferent worse than what we use now! p0p1

28. p. 28 A class of decision rules: Reject drug if # responses is... r1 out of n1 (first stage) (SHORT WALL) or... r out of n (full trial),(TALL WALL) A set of criteria: alpha = Type I error < 0.10, p0 = 0.30 beta = Type II error < 0.10. p1 = 0.50 Minimize the average sample size if p0 is true: E(N | p0) Optimal design : r1/n1 r/ n E(N | p0) 7/22 17/46 29.9

29. p. 29 This might be a “Type II error”. This might be a “Type I error”. =7 =17 =46 =22

30. Let’s play Phase II Trial! You are the Principal Investigator. p. 30

31. p. 31 Objective: Comparative, confirmatory analysis Endpoint: Clinically directly meaningful & important (Survival; Time to Progression; Symptom relief) Treatment assignment: Randomized Control: “Standard of care” usually Early stopping:Evidence for non-equivalence. - New treatment clearly better. - New treatment clearly worse. Phase III studies

32. p. 32 What is a “statistic”? “Test statistic” measures “surprise if the null hypothesis is true”. P-value = If P-value, “whoopee”. (“reject the null”) If not, then “poopie”. (“accept the null”)

33. p. 33 Strength of evidence: the “P-value” “P=0.01”: “In a long series of identical trials,if the null hypothesis is true”, such an unusual result as OUR study would only occur once in a hundred trials (0.01 of the time).” ordering of what is possible outcomes “unusual” { }  { }

34. p. 34 Survival Curves for Arms A and B

35. Let’s play Phase III Trial! You are a figment in statistician’s computer ( a SIMULATION). Except ONE of you is real. p. 35

36. p. 36 Historical controls can be misleading… This is one reason we randomize! Arm A = 5-FU and LV (1983 Trial) (Advanced Colorectal Cancer) Arm B = 5-FU and LV (1986 Trial) (Advanced Colorectal Cancer) Arm A is the better regimen!

37. Stuff we “knew” that ain’t so… (animal studies, observational “big data”, …) p. 37 Womens Health Initiative The unopposed estrogen trial was halted in February 2004, after an average follow- up period of 6.8 years, on the basis that unopposed estrogen does not appear to affect the risk of heart disease, the primary outcome, which was in contrast to the findings of previous observational studies. On the other hand, there were indications for an increased risk of stroke. The Effect of Vitamin E and Beta Carotene on the Incidence of Lung Cancer and Other Cancers in Male Smokers Unexpectedly, we observed a higher incidence of lung cancer among the men who received beta carotene than among those who did not. Randomized trials, hooray!!

38. p. 38 Advanced Colorectal Cancer SYMPTOM STATUS Maybe this explains why Arm A did better. Let’s do some statistical magic!

39. p. 39 Advanced Colorectal Cancer Cox Proportional Hazards Model “ Controlling for risk factors”

40. p. 40 Advanced Colorectal Cancer Cox Proportional Hazards Model

41. p. 41 DUKES’ B Confounding

42. p. 42 DUKES’ C Confounding

43. p. 43 ALL PATIENTS Confounding

44. p. 44 Confounding – “Simpson’s paradox”

45. Afterthoughts p. 45 New ideas in clinical trial design are growing rapidly! Clinical trials should become more ethical. “Personalized medicine”– a crisis looming. Explosion of “features”. People “like me” – fewer and fewer. Sample sizes smaller, but effect sizes bigger (we hope). The best discussion of Simpson’s Paradox is in Judea Pearl’s book, Causality, Chapter 6. A FUN READ!