1. Types of study designs: from descriptive studies to randomized controlled trials
Kirsten Bibbins-Domingo, PhD, MD
Assistant Professor of Medicine and of Epidemiology and Biostatistics
University of California, San Francisco

2. Objectives
To understand the difference between descriptive and analytic studies
To identify the strengths and weakness of different designs
To be able to apply different study designs to the same research question

3. Descriptive vs. Analytic
Test Performance
Height
Descriptive Questions
What is the average height of UCSF students?
How many UCSF student are over 6 feet tall?
Analytic Questions
Is student height associated with test scores?
Do students greater than 6 feet tall score higher on standardized tests?

4. Hierarchy of Study Types??
Analytic
Descriptive
Case report
Case series
Survey
Observational
Cross sectional
Case-control
Cohort studies
Experimental
Randomized
controlled trials
Strength of evidence for causality between a risk factor and outcome

5. Analytic Studies
Attempt to establish a causal link between a predictor/risk factor and an outcome.
You are doing an analytic study if you have any of the following words in your research question:
causes, leads to, compared with, more likely than, associated with, related to, similar to, correlated with, greater than, less than
Predictor
(risk factor)
Outcome
(disease)

6. Measures of association
Risk ratio (relative risk) A
A + B C
C + D
Disease
Yes No
Risk
Factor A B C D

7. Hierarchy of Study Types
Analytic
Descriptive
Case report
Case series
Survey
Observational
Cross sectional
Case-control
Cohort studies
Experimental
Randomized
controlled trials
Strength of evidence for causality between a risk factor and outcome

8. Research Question Is the regular consumption of Red Bull
associated with improved academic
performance among U.S. medical students?

9. Rationale
“functional drink” designed for periods of mental and physical exertion.
performance, concentration, memory, reaction time, vigilance, and emotional balance
Taurine + glucuronolactone + caffeine

10. Background
Alford C, Cox H, Wescott R. The effects of red bull energy drink on human performance and mood. Amino Acids. 2001;21(2):
Warburton DM, Bersellini E, Sweeney E. An evaluation of a caffeinated taurine drink on mood, memory and information processing in healthy volunteers without caffeine abstinence. Psychopharmacology (Berl) Nov;158(3):322-8.
Seidl R, Peyrl A, Nicham R, Hauser E. A taurine and caffeine-containing drink stimulates cognitive performance and well-being. Amino Acids. 2000;19(3-4):
Horne JA, Reyner LA. Beneficial effects of an "energy drink" given to sleepy drivers. Amino Acids. 2001;20(1):83-9.
Kennedy DO, Scholey AB. A glucose-caffeine 'energy drink' ameliorates subjective and performancedeficits during prolonged cognitive demand. Appetite Jun;42(3):331-3.

11. Great idea, but how do you get started….
Interesting, novel, and relevant, but…
You only have 25,000 dollars to start investigating this question.
What is feasible?

12. Study Design #1
Cross-sectional study of UCSF medical students taking USMLE Step 2
Questionnaire administered when registering for USMLE 2
Primary predictor: self-report of >3 cans Red Bull per week
Primary outcome: Score on USMLE Step 2

13. Cross-sectional study: structure
Predictor
(risk factor)
Outcome
(disease)
Red Bull consumption
USMLE Score
time

14. Cross-sectional Study: Pluses
+ Prevalence (not incidence)
+ Fast/Inexpensive - no waiting!
+ No loss to follow up
+ Associations can be studied
Many well-known cross-sectional studies
AAMC
California Health Interview Survey (NHIS, CHIS)
National Health and Nutrition Exam Survey (NHANES)

15. Cross-sectional study: minuses
- Cannot determine causality
Red Bull consumption
USMLE Score
time

16. Cross-sectional study: minuses
- Cannot determine causality
ACE inhibitor use hospitalization for heart failure
Documented DNR status in-hospital mortality
time

17. Cross-sectional study: minuses
- Cannot determine causality
- Cannot study rare outcomes

18. What if you are interested in the rare outcome?
The association between regular Red Bull consumption and…
A perfect score on the USMLE – Step 2
Acceptance into a highly selective residency
ANSWER: A Case-Control study

19. Study Design #2 A case-control study
Cases: 4th year med students accepted to residency in “highly selective specialty X”.
Controls: 4th year med students who applied but were not accepted.
Predictor: self-reported regular Red Bull consumption

20. Case control studies
Investigator works “backward” (from outcome to predictor)
Sample chosen on the basis of outcome (cases), plus comparison group (controls)
Predictor
(risk factor)
Outcome
(disease)

21. Case-control study structure
present
CASES
4th year UCSF students
who matched in “highly
selective specialty X”
Red Bull consumption
YES
Red Bull consumption NO
CONTROLS
4th year students who failed
to match in “highly selective
specialty X”
time

22. Case control studies
Cannot yield estimates of incidence or prevalence of disease in the population (why?)
Odds Ratio is statistics

23. Measures of association
Odds ratio
A D
C B
Disease
Yes No
Test A B C D

24. Case-control Study: pluses
+ Rare outcome/Long latent period
+ Inexpensive and efficient: may be only feasible option
+ Establishes association (Odds ratio)
+ Useful for generating hypotheses (multiple risk factors can be explored)

25. Case-control study-minuses
Causality still difficult to establish
Selection bias (appropriate controls)
Caffeine and Pancreatic cancer in the GI clinic
Recall bias: sampling (retrospective)
Abortion and risk of breast cancer in Sweden
Cannot tell about incidence or prevalence

26. Case-control - “the house red”
Rely tampons and toxic shock syndrome:
High rates of toxic shock syndrome in menstruating women
Suspected OCPs or meds for PMS
Cases: 180 women with TSS in 6 geographic areas
Controls: 180 female friends of these patients and 180 females in the same telephone code
Tampon associated with TSS (OR = 29!)
Super absorbency associated with TSS (OR 1.34 per gm increase in absorbency)
Led to “RELY” brand tampons being taken off the market.

27. Where are we?
Preliminary results from our cross-sectional and case-control study suggest an association between Red Bull consumption and improved academic performance among medical students
What’s missing? - strengthening evidence for a causal link between Red Bull consumption and academic performance
Use results from our previous studies to apply for funding for a prospective cohort study!

28. Study design #3
Prospective cohort study of UCSF medical students Class of 2009
All entering medical students surveyed regarding beverage consumption
Outcomes: USMLE Step 1 score, USMLE Step 2 score, match in first choice residency

29. Elements of a cohort study
Selection of sample from population
Measures predictor variables in sample
Follow population for period of time
Measure outcome variable
Famous cohort studies
Framingham
Nurses’ Health Study
Physicians’ Health Study
Olmsted County, Minnesota
Predictor
(risk factor)
Outcome
(disease)

30. Prospective cohort study structure
The present
The future
Top USMLE scorers
Everyone else
time

31. Strengths of cohort studies
Know that predictor variable was present before outcome variable occurred (some evidence of causality)
Directly measure incidence of a disease outcome
Can study multiple outcomes of a single exposure (RR is measure of association)

32. Weaknesses of cohort studies
Expensive and inefficient for studying rare outcomes
HERS vs. WHI
Often need long follow-up period or a very large population
CARDIA
Loss to follow-up can affect validity of findings
Framingham

33. Other types of cohort studies
Retrospective cohort
Identification of cohort, measurement of predictor variables, follow-up and measurement of outcomes have all occurred in the past
Much less costly than prospective cohorts
Investigator has minimal control over study design

34. Hierarchy of Study Types
Analytic
Descriptive
Case report
Case series
Survey
Observational
Cross sectional
Case-control
Cohort studies
Experimental
Randomized
controlled trials
Strength of evidence for causality between a risk factor and outcome

35. What distinguishes observational studies from experiments?
Ability to control for confounding
Confounder
Predictor
Outcome
Examples:
sex (men are more likely to drink red bull and men are more likely to match in neurosurgery)
Undergraduate institution (students from southern California are more likely to drink red bull and also score higher on USMLE)
Height?

36. But we measured all of the potential confounders…….
In a prospective cohort study you can (maybe) measure all potential known confounders, but…
You can’t control for unanticipated or unmeasured confounders
Randomization controls for unmeasured confounding

37. Study design # 4
Randomized controlled trial of daily Red Bull consumption among entering UCSF medical students Class 2009
Randomized to daily consumption of Red Bull vs. daily consumption of placebo
Outcomes: USMLE Step 1 score, USMLE Step 2 score, match in first choice residency

38. Steps in a randomized controlled trial
Select participants
high-risk for outcome (high incidence)
Likely to benefit and not be harmed
Likely to adhere
Measure baseline variables
Randomize
Eliminates baseline confounding

39. Steps in a randomized controlled trial
Blinding the intervention
Follow subjects
Adherence to protocol
Lost to follow up
Measure outcome
Clinically important measures
Adverse events

40. Why blind?: Co interventions
Unintended effective interventions
participants use other therapy or change behavior
study staff, medical providers, family or friends treat participants differently
Nondifferential - decreases power
Differential - causes bias

41. Why blind?: Biased Outcome Ascertainment or adjudication
If group assignment is known
participants may report symptoms or outcomes differently
physicians or investigators may elicit symptoms or outcomes differently
Study staff or adjudicators may classify similar events differently in treatment groups
Problematic with “soft” outcomes
investigator judgment
participant reported symptoms, scales

42. High Quality Randomized Trials
Tamper-proof randomization
Blinding of participants, study staff, lab staff, outcome ascertainment and adjudication
Adherence to study intervention and protocol
Complete follow-up

43. Why not just jump to an RCT?
Expensive
Feasibility concerns
Ethical considerations
May not be the right design for your question

44. Hierarchy of Study Types??
A study type of every budget,
purpose and research question
Analytic
Descriptive
Case report
Case series
Survey
Observational
Cross sectional
Case-control
Cohort studies
Experimental
Randomized
controlled trials
Strength of evidence for causality between a risk factor and outcome