Developed through the APTR Initiative to Enhance Prevention and Population Health Education in collaboration with the Brody School of Medicine at East Carolina University with funding from the Centers for Disease Control and Prevention Experimental Studies

APTR wishes to acknowledge the following individual that developed this module:  Jeffrey Bethel, PhD Department of Public Health Brody School of Medicine at East Carolina University This education module is made possible through the Centers for Disease Control and Prevention (CDC) and the Association for Prevention Teaching and Research (APTR) Cooperative Agreement, No. 5U50CD The module represents the opinions of the author(s) and does not necessarily represent the views of the Centers for Disease Control and Prevention or the Association for Prevention Teaching and Research.

1. Recognize use of experimental studies as an epidemiologic study design 2. Distinguish between types of experimental studies 3. Describe key features of conducting experimental studies 4. Recognize special considerations of experimental studies

 Experimental studies (experimental)  Researcher determines who is exposed (treatments received)  Cohort studies (observational)  Case-control studies (observational)  Cross-sectional studies (observational)

 Goal of public health and clinical medicine is to modify natural history of disease and improve morbidity and mortality  How do we select the best preventive and therapeutic measures?  Carry out studies to determine value of various measures

Smith, AH. The Epidemiologic Research Sequence. 1984

 Most closely resemble controlled laboratory experiments  Gold standard of epidemiological research  High status and validity and can pick up small and modest effects

 James Lind identified symptoms of scurvy among sailors at sea after as little as a month  Conducted early experimental study on treatment of scurvy in mid-1700’s among British sailors  Small sample size (6 groups of 2 ill sailors)  Group eating oranges and lemons were fit for duty in 6 days

 Evaluate new drugs and other treatments for diseases  Evaluate new medical and health care technology  Evaluate new screening programs or techniques  Evaluate new ways of organizing or delivering health services (e.g. home v. hospital care following myocardial infarction)

 Preventive  Does prophylactic agent given to healthy or high-risk individual to prevent disease?  Therapeutic  Does treatment given to diseased individual reduce risk of recurrence, improve survival, quality of life?

 Individual  Do women with stage I breast cancer given a lumpectomy alone survive as long without recurrence of disease as women given a lumpectomy plus radiation?  Community  Does fluoride in the water supply decrease the frequency of dental caries in a community compared to a similar community without such water treatment?

STUDY POPULATION CURRENT TREATMENT NEW TREATMENT IMPROVE DO NOT IMPROVE DO NOT IMPROVE RANDOM ASSIGNMENT

 Hypothesis formed  Participants recruited based on specific criteria and their informed consent is sought  Eligible and willing subjects randomly allocated to receive one of the two or more interventions being compared  Study groups are monitored for outcome under study (recurrence of disease, first occurrence of disease, getting better, side effects)  Rates of the outcome in the various groups are compared

 Women with stage I breast cancer given a lumpectomy alone will survive as long without recurrence of disease as women given a lumpectomy plus radiation  Water supply with fluoride will decrease the frequency of dental caries in a community compared to a similar community without water treated with fluoride

 Who will be in the study?  Must be defined specifically before study begins  Remove subjectivity  Reproducibility

 Women’s Health Study  ≥ 45 years  No history of coronary heart disease, cerebrovascular disease, cancer, or other major chronic illness  No history of side effects to any of study medications  Were not taking any of following meds more than once per week: aspirin, NSAIDs, supplements of vitamin A, E, or beta-carotene  Were not taking anticoagulants or corticosteroids NEJM 352;13:

 How many participants do we need to enroll in the study?  Programs and tables exist to calculate sample size based on various parameters

TRUTH IN THE POPULATION CONCLUSION FROM SAMPLE H o (no difference) H 1 (there is a difference) Fail to reject H o (no difference) Correct decisionType II error (Probability =  ) False negative Reject H o (there is a difference) Type I error (Probability =  ) False positive Correct decision (Probability = 1-  ) Type I and II errors can be reduced by increasing sample size

 The difference in effect to be detected  Estimate of effect in one group  Level of significance (   Probability of concluding treatments differ when they do not differ  Level of power desired (1 - β)  Probability of concluding treatments differ when they do differ  1-sided or 2-sided test

 Compare the outcome among “exposed” to what the outcome would have been if unexposed  This comparison is counterfactual  Instead, compare the outcome among “exposed” group to the outcome in a “substitute” population  Validity of inference depends on finding a valid substitute population

 Need to randomly assign participants to one of the intervention groups (test or control)  Randomization  Next assignment is unpredictable  Coin toss to determine group allocation  Random number table, opaque envelopes  Computer

 Main purpose  Reduces selection bias in the allocation of treatment  Each participant has an equal chance of being in test or control group  Secondary purpose  If large enough sample size, produce treatment and control groups with similar baseline characteristics  Control for known and unknown factors

Baseline Characteristics in a study of heart disease patients Characteristic Test Group (n = 9,599) Control Group (n = 9,586) Male (%)72 White (%)95 Current smoker (%)2930 Patients with a history of: Hypertension (%)5251 Stable angina (%)22 High cholesterol (%)41

Baseline Characteristics in a study Maternal-Infant HIV Transmission Characteristic Test Group (n = 239) Control Group (n = 238) Median age at entry (yrs)2425 White (%)4838 Gestational age at entry2930 Median (weeks) weeks (%)5250 > 26 weeks (%)4850 Median CD4 county at entry41

 Treatment  Keep track of which treatment group the participant was assigned  Keep track of which therapy received  Baseline data  Collect baseline demographic and other risk factor data  Compare treatment groups

 Measuring outcome  Must be conducted in same fashion for all treatment groups  Preventive studies ▪ Precursors of disease or first occurrence of disease  Therapeutic studies ▪ Symptom improvement ▪ Length of survival ▪ Disease recurrence

 Myocardial infarction  Symptoms met WHO criteria  Abnormal levels of cardiac enzymes or diagnostic electrocardiograms  Stroke  New neurologic deficit of sudden onset that persisted for at least 24 hours  Death from cardiovascular disease  Examination of autopsy reports, death certificates, medical records, and information obtained from the next of kin or other family members

 Masking (Blinding)  Prevents conscious and subconscious bias in research  Use placebo to mask  Single blind: participants do not know which treatment they are receiving  Double blind: participants and observers (data collectors) do not know participant treatment status

 Parallel  Participants in each group simultaneously receive one study treatment  Treatment and comparison groups consist of different participants  Crossover  Planned reversal of intervention and control groups  Each participant can serve as his/her own control

STUDY POPULATION NEW TREATMENT CURRENT TREATMENT RANDOMLY ASSIGNED Group 1 Group 2 Group 1 Observe and Measure Effects Observe and Measure Effects

 Simple  Each group receives a treatment consisting of one component (e.g. one drug)  Factorial  Use same study population to compare 2 or more treatments  2 x 2 factorial design  Similar to 3 arms (drug A, drug B, and placebo) with fewer participants

Drug A Drug B YesNo Efficacy of B Yes Both A and B (cell a) B only (cell b) a+b v. c+d No A only (cell c) Neither (cell d) Efficacy of A a+c v. b+d

Aspirin Beta- carotene YesNo Efficacy of Beta- Carotene Yes Aspirin and Beta-carotene (cell a) Beta-carotene only (cell b) a+b v. c+d No Aspirin only (cell c) Neither (cell d) Efficacy of Aspirin a+c v. b+d

 Overt  Notify investigators he/she is dropping out of study  Drop outs  Covert  Stop taking assigned treatment without telling investigators  Need to build compliance checks in to the study (e.g. test urine, count pills, etc.)

 Efficacy  Reduction in risk  Calculate risk of death, developing disease, complications in each group  Vaccine example = (Rate in placebo group) – (Rate in vaccine group) Rate in placebo group

 Relative risk  Kaplan-Meier plot  Hazard ratio  Number of patients who would need to be treated (NNT) to prevent 1 adverse event  Number needed to harm (NNH) indicates number patients treated to cause harm in 1 patient who would not otherwise have been harmed

 Internal validity  Extent to which the study groups are comparable  Comparability  Reflected by selection/randomization  External validity  Extent to which the results of a study can be applied to people not in it  Generalizability  Representativeness

STUDY POPULATION CURRENT TREATMENT NEW TREATMENT RANDOMLY ASSIGNED REFERENCE POPULATION External Validity Internal Validity

 Items affecting internal validity  Loss to follow-up  Lack of randomization  Items affecting external validity  Loss to follow-up  Low response rate  Narrow inclusion criteria

 Randomization  There must be genuine uncertainty about which treatment is better  Informed consent  Some trials enroll participants immediately after diagnosis  When to stop the study?  Harmful or beneficial effects of one treatment arm  Outside board monitors study

 Expensive and time-consuming  Ethical concerns may arise  A large number of participants may be required  Participant exclusion may limit generalizability  Compliance may be an issue  Influence of sponsorship

 Randomization tends to balance risk factors across study groups  Blinding of participants can reduce bias in assessment of outcomes  Prospective design  Eliminate bias by comparing two otherwise identical groups  Detailed information collected at baseline and throughout study period

 Experimental studies top epidemiologic study design hierarchy in terms of validity  Investigators assign treatment to participants (experimental)  Randomization reduces selection bias in treatment allocation  Data collection must be conducted systematically  Noncompliance and drop-outs must be minimized to increase validity of results

 Center for Public Health Continuing Education University at Albany School of Public Health  Department of Community & Family Medicine Duke University School of Medicine

Mike Barry, CAE Lorrie Basnight, MD Nancy Bennett, MD, MS Ruth Gaare Bernheim, JD, MPH Amber Berrian, MPH James Cawley, MPH, PA-C Jack Dillenberg, DDS, MPH Kristine Gebbie, RN, DrPH Asim Jani, MD, MPH, FACP Denise Koo, MD, MPH Suzanne Lazorick, MD, MPH Rika Maeshiro, MD, MPH Dan Mareck, MD Steve McCurdy, MD, MPH Susan M. Meyer, PhD Sallie Rixey, MD, MEd Nawraz Shawir, MBBS

 Sharon Hull, MD, MPH President  Allison L. Lewis Executive Director  O. Kent Nordvig, MEd Project Representative