1. ASPIRE CLASS 3: Choosing Your Study Design and Population
Kari L. Olson, BSc(Pharm), Pharm D, FCCP, BCPS (AQ Cardiology)

2. Learning Objectives Class 2: Study Designs and Population ASPIRE
Assess the advantages and disadvantages of various study designs,
Compose study population selection criteria for your study
Differentiate between internal and external validity and generalizability
bias and confounding

3. Elements of a Research Protocol
Background
Population
Study Design
Objectives
Procedures
Analytical Plan

4. Hierarchy of Evidence

5. Study Designs Bias $ Primary Study Designs Experimental Studies:
Randomized controlled trial
Non-randomized controlled trial
Observational Studies:
Cohort
Case-control
Before/after with no control
Cross-sectional
Case series/case reports $
Bias
One area to differentiate b/w experimental vs. observational studies. Efficacy studies, ie. demonstrating whether something is “efficacious” is conducted in controlled experimental research trial. A study that shows a treatment approach to be “efficacious” means that the study produced good outcomes, which were identified in advance, in a controlled experimental trial, often in highly constrained conditions. Phase III drug studies are good examples for efficacy trials.
Effective treatment provides positive results in a usual or routine care condition that may or may not be controlled for research purposes but may be controlled in the sense of specific activities are undertaken to increase the likelihood of positive results. Effectiveness studies use real-world clinicians and clients, and clients who have multiple diagnoses or needs. These can also be randomized, however the conditions are less constrained than that of efficacy trials. CER

6. Study Designs A few ways to do it... Exposure Disease/Outcome
Population at time X look at exposure and outcome

7. What is a cohort? Cohort: Two purposes:
Group of people who have something in common
Single group or multiple groups
Two purposes:
Descriptive
measures of frequency, incidence rates or describe natural history of disease
Analytic
associations between rates of outcomes, risk factors, or predictors of outcome

8. Cohort Studies Longitudinal, follow-up, or observational study
Identify subjects, then observe and record characteristics over time to measure outcomes
Retrospective, prospective or before/after (pre/post)
Time-order relationships clear in cohort studies

9. Observational Study: Prospective Cohort
Experimental
Exposure
No Exposure
With
outcome
Without
Subjects selected for the study
RANDOMIZATION
Without
outcome
With outcome
Today
Time

10. Example of Prospective Cohort Study
Study Objective: to determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with increases in all-cause and cardiovascular mortality
Jul ’97 to Jan ‘00
Today
3258 patients scheduled for coronary angiography enrolled; Vit D levels drawn
Outcomes:
All-Cause Mortality
Cardiovascular Mortality
Study
End
Lower Vit D
Quartiles
What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up
Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies.
Also, cohort studies are useful if you are determining risks where randomization may not be ethical.
Follow-up
7.7 yrs
Higher Vit D
Quartiles
All-cause: HR 2.1,
CV: HR 2.2,
Arch Intern Med 2008;168:

11. Cohort Study to Experimental Study
Study Objective: to determine if supplementing with Vitamin D
reduces all-cause and cardiovascular mortality among patients
undergoing coronary angiography
Follow-up
7.7 yrs
Jul ’97 to Jan ‘00
Today
3258 patients scheduled for coronary angiography enrolled; Vit D levels drawn
Lower Vit D
Quartiles
Higher Vit D
Outcomes:
All-Cause Mortality
Cardiovascular Mortality
Study
End
RANDOMIZE
Vit D or placebo
Arch Intern Med 2008;168:

12. Observational Study: Retrospective Cohort
Subjects selected for the study
Exposure
No Exposure
With
outcome
Without
Can look similar to a proscpective cohort, except that everything occurs in the past rather than prospective x
Time
Today

13. Example of Retrospective Cohort Study
Time x
Today
Lower Vit D
Quartiles
3258 patients scheduled for coronary angiography enrolled; Vit D levels drawn
Follow-up
7.7 yrs
Outcomes:
All-Cause Mortality
Cardiovascular Mortality
Higher Vit D
Quartiles
You could mimic the prospective cohort design we just discussed but change your time to today and look backwards. In this case, because the data have already been collected in the past but we are following pts in time (albeit retrospectively), this would be a longitudinal, retrospective, cohort study. Some problems with this is ensuring all pts have the required data, for instance Vit D levels. You would have to limit your sample to just those pts who had a Vit D level drawn which could introduce some bias in patient selection and thus limit interpretation of the outcome.
Jul ’97 to Jan ‘00
Arch Intern Med 2008;168:

14. Cohort Studies Advantages Disadvantages Can study multiple outcomes
Can study uncommon outcomes or outcomes associated with risk
Selection bias less likely
Gives you incidence data
Can be expensive
Can take time (prospective)

15. x Observational Design: Case-control Studies Today Exposure Cases Yes
Subjects with condition ('cases') matched with patients without condition (‘controls’)
Exposure
Yes No
Cases
Look back in time
Controls
Today x

16. Example of Case-Control Study
N Engl J Med 2000;343:
Study Objective: To determine the association between
phenylpropanolamine and hemorrhagic strokes
Exposed to phenylpropanolamine?
Cases: pts with hemorrhagic stroke
Yes No
Look back in time
Yes No
Controls: patients with no hemorrhagic stroke x
Today

17. Observational Designs: Case-control Studies
Advantages:
rare diseases,
short time periods,
inexpensive,
can study multiple potential causes
Disadvantages:
bias,
validation on exposure difficult,
does not provide incidence of disease,
choice of control group difficult,
difficult interpreting results

18. Observational Study: Pre/Post Cohort
No exposure to intervention
“Pre” Period
Exposure to
intervention
“Post” Period
Without
outcome
With
“Baseline”
Without
outcome
With
“Follow-up”
Subjects
identified
for study
Time
Time
Can be done prospectively or retrospectively, however given typically no comparator group, you probably want to

19. No exposure to intervention
Observational Study: Pre/Post Cohort
Study Objective: to assess the effect of an electronic prescribing (EP) system on the incidence and type of prescribing errors and the number of error-free visits
No exposure to intervention
Exposure to
intervention
Without
outcome
With
“Baseline”
Without
outcome
With
“Follow-up”
520 outpatients
in nephrology clinic in
tertiary pediatric hospital
Time
Time

20. Cohort Studies: Pre/Post Studies
Advantages
Disadvantages
Quick and relatively easy to implement
Hypothesis generating by which to design experimental studies
Typically, no control group by which to compare outcomes
Subject to bias
Regression to the mean

21. Observational Designs: Cross-sectional
Observation of a defined population at a single point in time or time interval
Advantages:
estimates of prevalence,
quick,
inexpensive
Disadvantages:
not controlled,
can not assess causality,
impractical for rare diseases

22. Observational Designs: Case-series/case reports
Association b/w observed effect and exposure based on detailed clinical evaluation
Advantages:
hypothesis generating
identify unusual situations
Disadvantages:
short term,
selection bias,
not controlled

23. Enhancing Causal Inferences in Observational Studies
Spurious Associations
Chance (random error)
limit by calculating sample size
Bias (systematic error)
Selection bias, measurement bias, surveillance, information (recall, interviewer, non-response), misclassification
Select appropriate population, predictor, and outcomes
Matching
Real Associations
Effect-cause
Confounding
Adjust outcome with variables that are different between groups
Observational studies allow to assess associations, not causality. Causality can only really be tested in randomized controlled designs. Again, the purpose of randomization is to control for known and unknown (unmeasurable) differences in the population. In observational studies, in which randomization has not occurred, there are a number of things to consider/address to help enhance causal inferences.
Effect-Cause: comparator groups, use time
Associated with the predictor variable and is a cause for the outcome

24. To determine the impact of smoking on lung cancer rates
Objective:
To determine the impact of smoking on lung cancer rates
Choose the most appropriate study design?
Randomized Controlled Study
Case Controlled Study
Cross-Sectional Study
Prospective Cohort Study

25. To describe osteonecrosis of the jaw associated with alendronate
Objective:
Choose the most appropriate study design?
Randomized Controlled Study
Case Series
Cross-Sectional Study
Prospective Cohort Study
To describe osteonecrosis of the jaw associated with alendronate

26. Writing Study Design in Protocol
Summarize in 1-sentence
Think about design issues, limitations of the design, and try to address them as you move through methods
Also state in this section the IRB requirements for your proposal

27. Elements of a Research Protocol
Background
Population
Study Design
Objectives
Procedures
Analytical Plan

28. Study Population Population: Sample: Target population: Study sample:
complete set of people with a specified set of characteristics
Sample:
subset of the population
Target population:
the population to which the results will be generalized
Study sample:
subset of the target population

29. Selecting the Study Population
1st step in narrowing down target population
Helps address to which population results will be generalizable
Should flow logically and directly from research question/hypothesis
Provide standards as to whether a patients should enter a study
No real guidelines on how to come up with these

30. Selection Criteria Determine target population Inclusion criteria
Clinical, demographic, socioeconomic characteristics
Inclusion criteria
What should subjects possess to be included
KPCO membership
Exclusion criteria
What would preclude participation?
High loss to follow-up, provide poor data, side effects/safety issue
Determine method to be used to assess eligibility
Who, what, how?
Methods of recruitment and by whom

31. Vulnerable Populations
Children
Pregnant Women
Elderly
Minorities
Students, employees
Non-english speakers
Terminally ill
Prisoners

32. Are my results valid and generalizable?
Validity (i.e. accuracy)
Is the study able to scientifically answer the questions it is intended to?
Internal validity
the degree to which conclusions about causal relationships can be made based
External validity
The degree to which results of a study can be true in other cases (i.e. different populations, settings)
Generalizability
Internal validity: based on the measures used, the research setting, and the research design, etc
External validity : If the same research study was conducted in those other cases, would it get the same results?.
A major factor in this is whether the study sample (e.g. the research participants) are representative of the general population along relevant dimensions.

33. Will your research produce valid findings?
Threats to Validity
Chance
Bias
Ways that the targeted and sample populations differ
Systematic or measurement errors
Not random errors
Confounding

34. Criteria to think about
How do the criteria affect internal and external validity of the study?
Are the criteria necessary for the safety of the participants?
Are the criteria ethical?
How will the criteria be determined?
Validity
Generalizability

35. Class 3 Assignment
Prepare a 5-10 min presentation for your small group session including draft versions of the following:
Study design and setting
Study population
inclusion and exclusion criteria
Are you including/excluding any vulnerable populations?
Be prepared to discuss generalizability and validity of your selection criteria
Please come prepared with the above items
September 3, 2:30-5:00
Kaiser Permanente Central Support Services