Randomized Trials: A Brief Overview Michael Porter July 20, 2004 Epi 590

Randomized Trial Comparative study between two or more interventions Exposure to the intervention is determined by random allocation AKA: RCT, experimental design

“Hierarchy” of Evidence Randomized Trial (experimental) Observational Analytic (e.g. case-control, cohort) Observational Descriptive (case series, descriptive analyses)

Agree to participate, Informed consent Eligible Ineligible Decline to participate Randomization Treatment A Treatment B Good outcome Poor outcome Population Eligibility criteria Recruitment The Basic Model

Strengths Highest level of evidence Everything can be specified up front Ideal scenario for inferential statistics Eliminates confounding

Weaknesses $$$ Time consuming Patients and providers must relinquish treatment decision to random chance Generalizability Subject to dropouts, crossovers, non-compliance Still vulnerable to bias

Two Broad Categories Pragmatic Explanatory attempt to simulate clinical realities more accurately during patient recruitment and during formulation of the randomly allocated treatment groups (effectiveness) Explanatory attempt to answer a more specific and narrow question- in order to maximize their ability to do this, eligibility criteria may seek a more homogenous set of patients (efficacy)

Bias Systematic error within the study that results in a mistaken estimate of the effect of therapy on disease Bias can be introduced into any step of the process, including enrollment, randomization, and assessment of outcomes

Internal validity The ability of a trial to come to the correct conclusion regarding the question being studied Determined by the protocol and execution of the trial

External validity The ability of a trial to produce results that are generalizable to the larger population of patients with the disease Determined by the eligibility criteria, the protocol, and the primary outcomes

Randomization The key step The biggest strength, and often the biggest challenge Randomization breaks the link between any unmeasured confounding variables and treatment status

Randomization 2 important elements concealment from the investigator unpredictability “Randomized trials appear to annoy human nature- if properly conducted, indeed they should”* Still vulnerable to selection bias *Schulz, K.F., Subverting randomization in controlled trials. JAMA, 1995. 274(18): p. 1456-8.

Subverting Randomization When researchers failed to adequately conceal randomization from the investigators, an average 41% percent increase in treatment effect occurred compared to trials where the randomization process was concealed appropriately Schulz, K.F., et al., Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA, 1995. 273(5): p. 408-12

Blinding Single blinded Double blinded Subject unaware of treatment assignment Double blinded Subject + evaluator unaware of treatment assignment Blinding provides protection against Placebo effect Observation bias

Sample Size (a.k.a. “power calculation”) Significance level α Type 1 error rate p-value Power 1-β 1-Type 2 error rate Variability of the outcome Degree of difference in outcomes

The “truth” Study conclusions Treatments do not differ Treatments differ Correct conclusion Type II Error (β) Type I Error (α) (1- β) Treatments do not differ Study conclusions 0.95 0.10 Treatments differ 0.90 0.05

Outcome Should only have one or two primary outcomes to which the study is powered Be careful of secondary analyses Multiple looks require adjustment of significance level

Time Treatment A (n=500) Outcome (n=500) Randomize (n=1000) Treatment B (n=500) Outcome (n=500)

? Time Treatment A (n=500) Outcome (n=600) Randomize (n=1000) Treatment B (n=500) Outcome (n=400)

Crossovers Analyze based on treatment received? Exclude crossovers from the analysis? Analyze based on initial random treatment assignment?

Intention to treat analysis analyze the outcomes based on the original randomized assignments, regardless of treatment actually received

Randomized study ≠ RCT Survey Community interventions Clinic interventions Be novel!

Ethical Issues Equipoise Data monitoring and safety committees Eligibility criteria

Summary RCT- the highest level of evidence Avoids confounding, but not necessarily bias Expensive, time consuming Randomization- preserve and protect it Blinded when possible Single important outcome Intention to treat Equipoise