1. Making Randomized Clinical Trials Seem Less Random Andrew P.J. Olson, MD Assistant Professor Departments of Medicine and Pediatrics University of Minnesota Medical School olso5714@umn.edu

2. Disclosures I have no financial interests to disclose. I will not discuss off label or investigational product use.

3. Learning Objectives Identify a framework for analyzing RCT’s in the learning environment Discuss major sources of bias in RCT’s Define validity and generalizability and be able to begin to assess these in real-world article. Use these skills in a small group environment

4. $

5. Agenda Overview of RCT’s Randomization Blinding Outcome measurement Analysis – Intention to treat?

6. Let’s start with a roadmap.

7. RCT Roadmap Population People at risk for heart attacks Sample 1000 Statins 1000 Placebo 50 heart attacks 25 heart attacks Randomization Treatment Outcome Follow-up

8. Randomization is Key By randomizing subjects to different groups, both known (measured) and unknown (unmeasured) variables should be randomly distributed. This controls for known and unknown confounding variables

9. What is Confounding? A confounding variable is associated with the receipt of treatment and the outcome. Statin trial: – Smoking, exercise, hypertension medications

10. Validity and Generalizability Validity: In the studied population, was the study performed in a way that the results are valid? Generalizability: Are these results applicable to my patients?

11. Elements of a Randomized Controlled Trial ElementBest Case Scenario (Described in paper) Validity or Generalizability Subject SelectionRecruitment Procedures and Entrance Criteria specified Generalizability RandomizationRandom Sequence Allocation Concealment Validity TreatmentFeasible, safe, delineatedGeneralizability

12. ElementBest Case Scenario (Described in paper) Validity or Generalizability Follow upComplete (all accounted for) and similar between groups Validity Co-interventionSame between groups and relevant co-interventions described Validity BlindingSubjects, Providers, and Outcome assessors Validity

13. ElementBest Case Scenario (Described in paper) Validity or Generalizability OutcomesMeasurable?Validity Meaningful?Generalizability Analysis and PowerIntention to treat?Validity Adequately powered?Validity Statistical Methods described and appropriate? Validity

14. Randomized Controlled Trials Overview of RCT’s Randomization Blinding Outcome measurement Analysis – Intention to treat?

15. Randomization Is the randomization of a subject to a group really random? – If allocation is truly random, it cannot be predicted – Random number table or generator – Examples of non-random allocation: Even or odd MRN Days of the week Morning or afternoon patients First patient the day

16. Randomization Allocation Concealment The sequence of allocation to different groups cannot be seen by subjects or providers Examples: – Sealed, opaque envelopes – Central voice-response system – Online systems

17. Randomized Controlled Trials Overview of RCT’s Randomization Blinding Outcome measurement Analysis – Intention to treat?

18. Blinding Ideally, the only difference between groups is the treatment (which no one knows about!) Triple Blinding is Ideal – No one knows the treatment group allocation Provider Subject Outcomes assessor Blinding protects against bias from: – Different receipt of co-interventions between groups – Differential outcome ascertainment

19. Co-interventions Population People at risk for heart attacks Sample 1000 Statins 1000 Placebo 50 heart attacks 25 heart attacks 60% take aspirin 30% take aspirin

20. Co-interventions By not knowing which group a subject is assigned to, subjects in different groups should be treated the same Neither those giving or receiving treatment know the assignment

21. Blinding in Treatment Studies

22. Blinding Population People at risk for heart attacks Sample 1000 Statins 1000 Placebo 50 heart attacks 25 heart attacks Randomization Treatment Outcome Follow-up In treatment studies, it is usually necessary to have a placebo or sham procedure

25. Blinding If subjects and providers are unaware of which group the patient is allocated to, co- interventions should be the same on average Differences in co-interventions, if there is proper randomization and blinding, will be due to chance.

26. Don’t forget about the third blind team member! Subjects and Providers can be difficult to blind, especially with certain treatments However, those who are analyzing the outcomes can almost always be blinded – Analysis of medical records – If they know the group assignment, their view of an outcome can be biased

27. Randomization and Blinding Population Sample Treatment A Treatment B # Events Randomization Treatment Outcome Follow-up Similar at baseline?Similar during followup?

28. Randomized Controlled Trials Overview of RCT’s Randomization Blinding Outcome measurement Analysis – Intention to treat?

29. Outcomes Outcomes are prespecified – Measurable - Validity – Meaningful – Generalizability Easily measurable: – Mortality, MI, cancer recurrence, blood pressure, lipids Less easily measured: – Quality of life, pain, disability – Validated tool? Meaningful: – Do they matter to a patient?

30. Surrogate Outcomes Sometimes a meaningful outcome is difficult to measure: – Time for followup, hard to quantify So a surrogate outcome is used: – FDA Definition: A laboratory measurement or physical sign that is used as a substitute for a clinically meaningful outcome because it is expected to predict the effect of therapy on a clinically meaningful outcome.

31. Surrogate Outcome Population Sample Treatment A Treatment B # Events Surrogate

32. Surrogate Outcome Treatment No Change Events ↑Surrogate ↓Surrogate ↑Harm

33. An Example of Surrogate Outcomes There is significant mortality from arrhythmias after myocardial infarctions PVC’s can be a marker of arrhythmias Antiarrhythmic medications decrease PVC’s Thus, it makes sense that using antiarrhythmic medications after myocardial infarctions might decrease mortality

34. A Classic Example of Surrogate Outcomes CAST Trial

35. Cardiac Mortality

36. All Cause Mortality

37. Randomized Controlled Trials Overview of RCT’s Randomization Blinding Outcome measurement Analysis – Intention to treat?

38. Intention to Treat Analysis 5000 Patients Screened 1000 Randomized 500 Placebo500 Metoprolol Outcome 23 withdraw consent 14 lost to followup 22 stop taking medicine 10withdraw consent 4 lost to followup 13 stop taking medicine ?

39. Intention to Treat All randomized subjects are included in the analysis, regardless of actual receipt of treatment This means some subjects who didn’t get the intervention are still included in the analysis Preserves the randomization

40. Intention to Treat All subjects should be able to be accounted for while you read the paper – High rate of participation – Few are “lost to followup” If a subject is lost to followup: – Search for vital statistics – Perform advanced analyses to determine what probably happened to these subjects Most importantly, patients must NOT be removed from the study in a non-random way!

41. Small Group Activity

42. Was the assignment of patients to treatments randomized? Were the groups similar at the start of the trial? Except for the allocated treatment, were the groups treated equally? Were all patients who entered the trial accounted for and were they analyzed in the groups to which they were randomized? Were the measures objective? Were the patients and clinicians kept blind to which treatment was being received? How large was the treatment effect? How precise was the estimate of the treatment effect? Will the results help me in caring for my patients?