Introduction to Randomized Clinical Trials Stephen Bent Associate Professor of Medicine University of California, San Francisco

Randomized Trials –Rationale –Basic designs –Participants –Intervention –Blinding –Outcomes –Adherence –Follow-up

Take Home Messages 1. RCT’s really only do one thing1. RCT’s really only do one thing –But, it’s a very important thing. 2. RCT’s are the BEST study design for evaluating efficacy.2. RCT’s are the BEST study design for evaluating efficacy. RCT’s compare mean responsesRCT’s compare mean responses –Not always what we’d like to know RCT’s take a long time and cost $RCT’s take a long time and cost $

Take Home Messages What is bias?What is bias? What is confounding?What is confounding?

Rationale Why do a randomized blinded trialWhy do a randomized blinded trial –minimize confounding!!! –minimize co-interventions –minimize biased outcome ascertainment Why not do a randomized trialWhy not do a randomized trial –major ethical issues –narrow research question –expensive –long time from idea to paper Generally reserved for mature questionsGenerally reserved for mature questions

Basic Trial Design Population Sample Intervention Randomization OutcomeOutcome ControlControl OutcomeOutcome + Blinding Placebo

Randomization Participants are assigned to treatment groups by chance with a known probabilityParticipants are assigned to treatment groups by chance with a known probability Random number table or computerRandom number table or computer Tamper-proof systemTamper-proof system –ordered, sealed envelopes –centralized system (phone, fax, web)

Value of Randomization Balances baseline characteristics of the treatment groupsBalances baseline characteristics of the treatment groups –eliminates confounding due to measured and unmeasured factors –provides an unbiased comparison between groups Does NOT maintain balance after randomizationDoes NOT maintain balance after randomization

Variations of Randomization Fixed Allocation - probability fixedFixed Allocation - probability fixed –Simple - flip a coin –Blocked - randomize consecutive small batches –Stratified - separate randomization in strata –Clustered - randomize groups Adaptive - probability changesAdaptive - probability changes

Cross-over Design Population Sample Intervention Randomization Placebo Washout Placebo Intervention Outcome

Factorial Design Population Sample Int A and Int B Int A and Pbo B Pbo A and Int B Pbo A and Pbo B Outcome

Participants Inclusion criteria to maximize:Inclusion criteria to maximize: –rate of outcomes (old, weak) –likely benefit from intervention –generalizability –ease of recruitment

Exclusion Criteria Intervention unsafeIntervention unsafe Intervention unlikely to be effectiveIntervention unlikely to be effective Unlikely to adhere to the interventionUnlikely to adhere to the intervention –Run in Unlikely to complete follow-upUnlikely to complete follow-up Practical problemsPractical problems Practice Parsimony Preserve Generalizability Practice Parsimony Preserve Generalizability

Choice of Intervention MaximizeMaximize –effectiveness (highest tolerable dose) –safety (lowest effective dose) –generalizability –trial design/conduct recruitmentrecruitment compliancecompliance blindingblinding

Choice of Control Inert placebo usually bestInert placebo usually best Active therapy or “standard of care”Active therapy or “standard of care”

Cointerventions Unintended effective interventionsUnintended effective interventions –participants use other therapy or change behavior –study staff, medical providers, family or friends treat participants differently Nondifferential - decreases powerNondifferential - decreases power Differential - causes biasDifferential - causes bias

Biased Outcome Ascertainment If group assignment is knownIf group assignment is known –participants may report symptoms or outcomes differently –physicians or investigators may elicit symptoms or outcomes differently If group assignment is knownIf group assignment is known –participants may report symptoms or outcomes differently –physicians or investigators may elicit symptoms or outcomes differently

Canadian Cooperative MS Trial 165 patients with multiple sclerosis165 patients with multiple sclerosis –plasma exchange + cyclo + pred –sham plasma exchange + placebo meds Outcome = structured neurologic exam by blinded and unblinded neurologistsOutcome = structured neurologic exam by blinded and unblinded neurologists More improvement with plasma exchange by unblinded, but not blinded assessmentMore improvement with plasma exchange by unblinded, but not blinded assessment 165 patients with multiple sclerosis165 patients with multiple sclerosis –plasma exchange + cyclo + pred –sham plasma exchange + placebo meds Outcome = structured neurologic exam by blinded and unblinded neurologistsOutcome = structured neurologic exam by blinded and unblinded neurologists More improvement with plasma exchange by unblinded, but not blinded assessmentMore improvement with plasma exchange by unblinded, but not blinded assessment Noseworthy, Neurology, 1994

Biased Outcome Adjudication Study staff who decide if a change or outcome has occurred mayStudy staff who decide if a change or outcome has occurred may –classify similar events differently in treatment groups Problematic with “soft” outcomesProblematic with “soft” outcomes –investigator judgement –participant reported symptoms, scales

Why Not Blind? ImpossibleImpossible –surgery –exercise –diet –education Possible, butPossible, but –dangerous –painful –cumbersome

Is It Really Blinded? Difficult even for drugsDifficult even for drugs –identical placebo difficult to prepare –drug may smell, taste, feel different –drug may cause side effects –test results may unblind –participants may test drug Difficult even for drugsDifficult even for drugs –identical placebo difficult to prepare –drug may smell, taste, feel different –drug may cause side effects –test results may unblind –participants may test drug

What if You “Can’t” Blind? Do the best you can –minimize differential cointervention –blind those measuring outcome –use “hard” outcomes Measure degree of unblinding

Be Courageous Laparoscopic lysis of adhesions for pelvic painLaparoscopic lysis of adhesions for pelvic pain Internal mammary ligation for anginaInternal mammary ligation for angina Orthoscopic debridement for OAOrthoscopic debridement for OA Sham burr holes for fetal tissue implants for Parkinson’sSham burr holes for fetal tissue implants for Parkinson’s

Do the Best You Can Exercise to prevent coronary eventsExercise to prevent coronary events –exercise - supervised exercise to 80% maximum capacity 30 min 3/wk –control - supervised exercise to 40% maximum capacity 30 min 3/wk Psychotherapy for schizophreniaPsychotherapy for schizophrenia –therapy - psychotherapy weekly –control - advice about diet, exercise, and smoking weekly

Use a “Hard” Outcome DeathDeath MeasurementsMeasurements –test results MVO 2 vs.. self-reported exercise abilityMVO 2 vs.. self-reported exercise ability Doppler evaluation vs.. swollen leg for DVTDoppler evaluation vs.. swollen leg for DVT –scales and diaries vs. investigator judgment Geriatric Depression Scale vs. “improved”Geriatric Depression Scale vs. “improved” 7-day urinary diary vs. “dry”7-day urinary diary vs. “dry”

Adherence Intervention cannot work if it isn’t usedIntervention cannot work if it isn’t used Adherence measuresAdherence measures –intervention pill count, diaries, biologic measure, measuring device in dispenserpill count, diaries, biologic measure, measuring device in dispenser –visits –study measurements

Women’s Health Initiative RQ: Does calcium plus vitamin D reduce risk of fractures in postmenopausal women? Design: Randomized trial Subjects: 36,282 PM women enrolled in WHI Intervention: 1 gm calcium IU vitamin D Outcome: clinical fractures Adherence at end of trial 60% and about 60% of placebo group was taking calcium

Outcomes in Clinical Trials Efficacy OutcomesEfficacy Outcomes –Primary –Secondary –Surrogate –Composite Adverse EffectsAdverse Effects –rare –common

Adverse Events and Side Effects AnticipatedAnticipated –use specific questions UnanticipatedUnanticipated –ask about general adverse experiences RareRare –sample size inadequate CommonCommon –multiple differences between groups

A Randomized Controlled Trial of a Chinese Herbal Remedy to Increase Energy, Memory, Sexual Function, and Quality of Life in Elderly Adults in Beijing, China Stephen Bent, MD Osher Center for Integrative Medicine University of California, San Francisco

Longevity Treasure Proprietary extractProprietary extract 10 Chinese herbs10 Chinese herbs Believed to increase “Yang”Believed to increase “Yang” Marketed to improveMarketed to improve –Energy, Memory, Sexual Function, Qi Widespread use in ChinaWidespread use in China US sales of over $1 millionUS sales of over $1 million

Research Question Does the daily use of Longevity Treasure lead to changes in energy, memory, sexual function, qi, or quality of life?Does the daily use of Longevity Treasure lead to changes in energy, memory, sexual function, qi, or quality of life?

Methods Design: Randomized Controlled TrialDesign: Randomized Controlled Trial ParticipantsParticipants –Chinese residents of Beijing, age > 60 –Self-reported decreased energy, memory, or sexual interest Recruitment – “word of mouth”Recruitment – “word of mouth”

Exclusion Criteria High YangHigh Yang Serious medical illnessSerious medical illness Currently taking Longevity TreasureCurrently taking Longevity Treasure

Intervention Random assignment toRandom assignment to –Longevity Treasure, 4 capsules three times a day (30 days) –Identical placebo, 4 capsules, three times a day (30 days)

Outcomes Assessed at baseline and 30 daysAssessed at baseline and 30 days PrimaryPrimary –Change in quality of life, SF-12 questionnaire –Change in quality of life, Qi scale

Secondary Outcome Measures Energy – questionnaireEnergy – questionnaire Energy – physical testsEnergy – physical tests –6 minute walk –Step test –Grip strength –Chair stands –Foot tap Memory – word and picture recallMemory – word and picture recall Sexual function – 3-item questionnaireSexual function – 3-item questionnaire

Study Flow

Baseline Characteristics CharacteristicPlaceboHerbsP-value Age Women (%) Decreased energy (%) Decreased memory (%) Decr. sexual interest (%) Qi score

Results: Primary Outcomes SF-12, Mental Component (Baseline = 53)SF-12, Mental Component (Baseline = 53) –Herbs: –Placebo: +2.5 –Difference: +1.9 points (95% CI, 0.1 to 3.6) SF-12, Physical Component (Baseline = 50)SF-12, Physical Component (Baseline = 50) –Herbs: +1.6 –Placebo: +1.7 –Difference: -0.1 (95% CI, -1.7 to 1.5)

Primary Outcome: Qi scale

Secondary Outcome Measures Measure (range)BaselineHerbPlaceboP-value Memory (0-39) Physical test (-16 to 8) Sexual function (0-26) SF-36 Vitality SF-36 Mental Health

Adverse Events EventPlacebo (%)Herb (%)P-value Diarrhea3.4 %1.7%0.41 URI1% 1.00 Headache0%1%0.31 Dry mouth5.9%12.7%0.07* Total19%29%0.24

Summary Longevity Treasure may improve mental healthLongevity Treasure may improve mental health –2 point increase on SF-12 mental health score –Similar improvement on SF-36 subscale The benefit, if any, is smallThe benefit, if any, is small Longevity Treasure does not improveLongevity Treasure does not improve –Memory –Sexual Function –Energy or Qi

Conclusions Longevity Treasure cannot be strongly recommended without further supportive evidenceLongevity Treasure cannot be strongly recommended without further supportive evidence RCTs of Chinese herbs are feasibleRCTs of Chinese herbs are feasible More work is needed to explore Chinese concepts of quality of life and qiMore work is needed to explore Chinese concepts of quality of life and qi

A randomized controlled trial of saw palmetto for the treatment of benign prostatic hyperplasia Stephen Bent, MD Christopher Kane, MD Katsuto Shinohara, MD John Neuhaus, PhD Harley Goldberg, DO Andrew L. Avins, MD, MPH University of California, San Francisco Kaiser Permanente Northern California, Division of Research Funded by the National Institute of Diabetes, Digestive and Kidney Diseases (NIDDK) and the National Center for Complementary and Alternative Medicines (NCCAM)

Saw palmetto (serenoa repens) Many patients seek an alternative to drug therapyMany patients seek an alternative to drug therapy Used daily by 1.1% of the US adult populationUsed daily by 1.1% of the US adult population Widely used in EuropeWidely used in Europe

Methods Randomized, double-blind, placebo-controlled trialRandomized, double-blind, placebo-controlled trial Patients recruited from Kaiser Permanente, Northern California and San Francisco VAMCPatients recruited from Kaiser Permanente, Northern California and San Francisco VAMC –Inclusion: Age > 50Age > 50 AUASI > 8 (range 0-35)AUASI > 8 (range 0-35) Peak flow > 4 and 4 and < 15 mls/sec –Exclusion: Prior prostate cancer or prostate surgeryPrior prostate cancer or prostate surgery Using alpha-antagonist, 5-ARI, or saw palmetto (washout permitted)Using alpha-antagonist, 5-ARI, or saw palmetto (washout permitted) Severe concomitant illnessSevere concomitant illness

Interventions Saw palmetto 160mg bid for one yearSaw palmetto 160mg bid for one year –Indena (extract, 92% free fatty acids) –Cardinal Health (encapsulation) –Rexall Sundown (packaging) Placebo: polyethylene glycol-400 and brown coloring agentPlacebo: polyethylene glycol-400 and brown coloring agent

STEP Study Flow Chart Saw palmetto 160 mg twice a day Placebo capsule twice a day 12 mo. 6 mo. 9 mo. 3 mo. 1 mo. 0 wks.-4 wks. -6 wks. Timeline Washou t Run-in Visit Enrollment Visit Randomized Treatment Period Visits Randomizatio n Visit Screening Period Visits

Baseline Characteristics CharacteristicSaw Palmetto (N=112) Placebo (N=113) Age (yr) White84%79% AUA Symptom Index Prostate Volume (ml) Maximal Urinary Flow Rate (ml/s) Serum PSA (ng/dl)1.81.6

Results: AUASI p = AUASI Month Saw palmettoPlacebo Randomization

Results: Maximum Flow Rate p = Maximum Urine Flow Rate Month Saw PalmettoPlacebo Randomization

Conclusions No evidence of benefit in AUASI, Qmax, or other measured outcomeNo evidence of benefit in AUASI, Qmax, or other measured outcome No evidence of harm; no safety concernsNo evidence of harm; no safety concerns Discrepancy between our findings and others’ results needs explorationDiscrepancy between our findings and others’ results needs exploration

Internet-Based RCT’s Exciting potentialExciting potential Reach unlimited # of patientsReach unlimited # of patients Lower costLower cost Much shorter time to completionMuch shorter time to completion Participate from home, convenientParticipate from home, convenient

Kava and Valerian for Anxiety and Insomnia: An Internet-based Randomized Blinded Trial

Figure 1.

REMOTE Study Tolterodine ER (Detrol) for OABTolterodine ER (Detrol) for OAB 1 st Internet-based RCT of drug1 st Internet-based RCT of drug Great idea, but many barriers currentlyGreat idea, but many barriers currently

Take Home Messages 1. RCT’s really only do one thing1. RCT’s really only do one thing –But, it’s a very important thing. 2. RCT’s are the BEST study design for evaluating efficacy.2. RCT’s are the BEST study design for evaluating efficacy. RCT’s compare mean responsesRCT’s compare mean responses –Not always what we’d like to know RCT’s take a long time and cost $RCT’s take a long time and cost $

Take Home Messages Bias = a systematic deviation of an observation from the true clinical stateBias = a systematic deviation of an observation from the true clinical state Confounder = a variable that is related to the predictor and a cause of the outcomeConfounder = a variable that is related to the predictor and a cause of the outcome