1. Design of Clinical Research Studies ASAP Session by: Robert McCarter, ScD Dir. Biostatistics and Informatics, CNMC rmccarte@cnmc.org

2. Topics What is study design? What is study design? What are the elements of study design? What are the elements of study design? What are threats to study validity that impact design? What are threats to study validity that impact design? What are the classic designs? What are the classic designs? How do you choose the right design? How do you choose the right design?

3. What is Study Design? Where does it fit in the scheme of things? Where does it fit in the scheme of things? From the 5000 foot level From the 5000 foot level Research Question Study Design Data Collection Data Analysis Design does not exist in the absence of a research question, ie. study aims Design does not exist in the absence of a research question, ie. study aims The clearer the aims the cleaner the design The clearer the aims the cleaner the design

4. Some early advice to promote Better Designs Keep your research questions simple Keep your research questions simple Focus on one Focus on one The best designs are centered on one clear and simple question The best designs are centered on one clear and simple question Clearer answers come from simple questions Clearer answers come from simple questions Does this mean you can only have one aim? Does this mean you can only have one aim?

5. What are the elements of study design? Definition Definition Structure Structure

6. Definition Clarify the purpose of the study Clarify the purpose of the study Describe Describe Compare / Relate Compare / Relate Precisely and Operationally define the elements of the research question Precisely and Operationally define the elements of the research question Define the Target Population Define the Target Population Define Study Group(s) Define Study Group(s) Define Study Outcome(s) Define Study Outcome(s)

7. The Research Paradigm Study Sample Result Target Population (Association) Conclusion About Scientific Theory (Causation) Statistical Inference Biological Inference

8. Paradigm-Design Connection Design concerns itself with Design concerns itself with Defining a Target population Defining a Target population At risk of the Study Outcome At risk of the Study Outcome Selection of a study sample Selection of a study sample Nonbiased representative sample to study Nonbiased representative sample to study Chosen at random Chosen at random

9. Is there an association? Is there bias? Possible explanations Confounding ChanceCausal Result Interpretation and Threats to Study Validity YesNo YesNo

10. Bias Bias Systematic error Invalid Results == Bias arises when a sample produces a result that is different from the result that would be obtained if the entire target population were studied. Bias arises when a sample produces a result that is different from the result that would be obtained if the entire target population were studied. Bias results arise from Biased Samples or Unequal Procedures Bias results arise from Biased Samples or Unequal Procedures The best protection against bias is to pay close attention to sampling procedures, to define and apply the study protocol and data collection procedures evenly across study groups. The best protection against bias is to pay close attention to sampling procedures, to define and apply the study protocol and data collection procedures evenly across study groups.

11. Some Types of Bias Selection bias Selection bias self-selection, e.g., volunteers self-selection, e.g., volunteers selective losses or nonresponse selective losses or nonresponse Information bias Information bias differential misclassification differential misclassification e.g., knowledge of risk factor status alters assessment of disease status or vice versa e.g., knowledge of risk factor status alters assessment of disease status or vice versa

12. Chance: Random Measurement Error Variation in measurements due to chance Variation in measurements due to chance Dealt with by statistical testing Dealt with by statistical testing P-value = the probability of seeing results as extreme or more extreme as those observed accounting for measurement error. P-value = the probability of seeing results as extreme or more extreme as those observed accounting for measurement error. Nominally, if the p-value is < 0.05 (1 chance in 20) we typically consider the results to be statistically significant Nominally, if the p-value is < 0.05 (1 chance in 20) we typically consider the results to be statistically significant more than can be accounted for by chance more than can be accounted for by chance

13. What is a Confounder? A variable entangled with the study factor that masks the true relationship between the study factor and the study outcome A variable entangled with the study factor that masks the true relationship between the study factor and the study outcome e.g., in comparing mortality in 2 groups that differ on a study factor, they may also differ in age e.g., in comparing mortality in 2 groups that differ on a study factor, they may also differ in age age is a very strong predictor of mortality and thus will confound comparisons between groups if the age difference is not addressed. age is a very strong predictor of mortality and thus will confound comparisons between groups if the age difference is not addressed.

14. Design Criteria Study Designs are Dictated by Study Aims There are really only 2 types of aims There are really only 2 types of aims Descriptive Aim Descriptive Aim e.g. Estimate the average rate of ear infections in a population of children > 95%tile for weight. e.g. Estimate the average rate of ear infections in a population of children > 95%tile for weight. Comparative Aim Comparative Aim e.g. Children > 95%tile for weight have more ear infections than children who are 95%tile for weight have more ear infections than children who are < 95%tile for weight.

15. Design for Comparative Aim- Controlled Study Random Sample >95%tile for weight Target Population DC Children Assess Frequency of Ear Infections in 1 year Random Sample <95%tile for weight Assess Frequency of Ear Infections in 1 year Study Group Control Group Compare Exposure Outcome

16. Keys to Controlled Study Design A Control group is essential to investigate associations A Control group is essential to investigate associations Like the Study Group except for ‘exposure’ of interest Like the Study Group except for ‘exposure’ of interest Controls should not be restricted to Normals Controls should not be restricted to Normals The appropriate control depends on the research question The appropriate control depends on the research question Groups differ on one and only one thing Groups differ on one and only one thing e.g., weight e.g., weight Otherwise, groups are formed & treated identically Otherwise, groups are formed & treated identically chosen from respective components of same pool chosen from respective components of same pool subjected to the same research protocol subjected to the same research protocol assessed equally for study outcome, e.g., ear infection assessed equally for study outcome, e.g., ear infection This allows the study to infer that any difference in the outcome between groups is likely due to the one difference allowed to exist between groups, e.g., weight. This allows the study to infer that any difference in the outcome between groups is likely due to the one difference allowed to exist between groups, e.g., weight.

17. Structure Determine study type – classic types Determine study type – classic types Prospective vs Retrospective Prospective vs Retrospective Observational vs. Experimental Observational vs. Experimental Observational = passive observer of natural events Observational = passive observer of natural events Experiment = manipulate events Experiment = manipulate events

18. Relationship between Natural History of Disease and Study Types Relationship between Natural History of Disease and Study Types “Risk” Factors Disease Natural History Cohort/RCT Case-Control Cross-sectional

19. Controlled Study Types in Order of Increasing Strength of Inference Case-Control Design Case-Control Design Cross-sectional Design Cross-sectional Design Cohort Design Cohort Design Randomized Clinical Trial (RCT), Experimental Design Randomized Clinical Trial (RCT), Experimental Design

20. Now lets look at each study design in some detail.

21. Case-Control Design Design in Words Design in Words Select representative cases (w/ study outcome) Select representative cases (w/ study outcome) Select comparable controls (w/o study outcome) Select comparable controls (w/o study outcome) Look historically in both groups for exposure Look historically in both groups for exposure Compare frequency of exposure in cases & controls Compare frequency of exposure in cases & controls

22. Case/Control Design Target Population Cases Non-Cases “Controls” Exposure Present Exposure Absent Exposure Present Exposure Absent

23. Interpretation of Case-Control Study All else being equal, if the frequency of exposure differs between the cases and controls by more than chance variation, the exposure is said to be related to the outcome. All else being equal, if the frequency of exposure differs between the cases and controls by more than chance variation, the exposure is said to be related to the outcome.

24. Special areas of vulnerability Case-Control Design Biased selection of Controls Biased selection of Controls Hard to ensure controls are like cases except for case status and representative of the population from which cases arose Hard to ensure controls are like cases except for case status and representative of the population from which cases arose Remedy – choose more than one control group Remedy – choose more than one control group Recall bias Recall bias Knowledge of outcome may bias determination of exposure Knowledge of outcome may bias determination of exposure Remedy – assess exposure before outcome is known Remedy – assess exposure before outcome is known

25. Cohort Study Design Design in Words Design in Words Select representative persons w/ study exposure Select representative persons w/ study exposure Select comparable controls w/o study exposure Select comparable controls w/o study exposure Follow equally to assess who develops outcome Follow equally to assess who develops outcome Compare risk of outcome in exposed & controls Compare risk of outcome in exposed & controls

26. Cohort Design Diagram Target Population Exposed Non-Exposed “Controls” Disease Present Disease Absent Disease Present Disease Absent Follow-up

27. Interpretation of Cohort Study All else being equal, if the risk of the outcome differs between the exposed and nonexposed by more than chance variation, the exposure is said to be related to the outcome. All else being equal, if the risk of the outcome differs between the exposed and nonexposed by more than chance variation, the exposure is said to be related to the outcome.

28. Special areas of vulnerability Cohort Design Unequal follow-up Unequal follow-up Hard to ensure equal follow-up of exposed and control Hard to ensure equal follow-up of exposed and control Remedy – minimize losses, account for follow-up variability in analysis Remedy – minimize losses, account for follow-up variability in analysis

29. Randomized Controlled Trial Design Design in Words Design in Words Select eligible persons from the population Select eligible persons from the population Randomly allocate them to receive or not receive intervention Randomly allocate them to receive or not receive intervention Follow equally to assess outcome Follow equally to assess outcome Compare risk or amount of outcome in intervention & control Compare risk or amount of outcome in intervention & control using intention-to-treat paradigm using intention-to-treat paradigm

30. RCT Design Diagram Alike except for Intervention Intervention NonIntervention Outcome Present Outcome Absent Outcome Present Outcome Absent Reference Population Randomize

31. Interpretation of RCT If the risk of the outcome differs between the intervention and nonintervention groups by more than chance variation, the intervention is said to affect the outcome If the risk of the outcome differs between the intervention and nonintervention groups by more than chance variation, the intervention is said to affect the outcome Provides the strongest evidence of cause Provides the strongest evidence of cause causal inference is sometimes based on one or two experiments causal inference is sometimes based on one or two experiments

32. Special areas of vulnerability Experimental Design (RCT) Share Follow-up Vulnerability with Cohort Study Share Follow-up Vulnerability with Cohort Study Generalizability Generalizability RCT enrollees are highly selective and may not be representative of the affected population RCT enrollees are highly selective and may not be representative of the affected population Yields estimates of the best result Yields estimates of the best result Remedy – follow the trial with community intervention study Remedy – follow the trial with community intervention study

33. Choosing a study design Consider a Case/Control or Cross-sectional study when investigating a completely new association, especially when the outcome is rare Consider a Case/Control or Cross-sectional study when investigating a completely new association, especially when the outcome is rare Consider a Cohort study for confirming associations uncovered above, especially to study exposures not amenable to an RCT or when the outcome is not rare Consider a Cohort study for confirming associations uncovered above, especially to study exposures not amenable to an RCT or when the outcome is not rare Choose a clinical trial to evaluate interventions Choose a clinical trial to evaluate interventions