1. Teaching Registrars Research Methods Study design
Landon Myer PhD
Senior Lecturer, Infectious Diseases Epidemiology Unit, School of Public Health, UCT

2. Orientation to today’s session
So far:
Introduction
Study protocol
Reviewing the Literature
Today: → Study design ←
To come:
Sampling
Measurement
Data analysis
Ethics
All of one interrelated
process

3. Overview How to start to select a study design
Framework for considering different study designs
Key similarities & differences
Introduction to each major category of study design
Focus on major functional features
Strengths & limitations
Examples
Exercises
Ask questions throughout!

4. Note on terminology “Outcomes” “Exposures”
Health outcome of interest in the study
Disease, death, side effect, complication
(stats: “dependent variables”)
“Exposures”
Measures that may be associated with the outcome
Possible “risk factors”, “causes”, “determinants”
(stats: “independent variables”)

5. I. Framework for thinking about study designs

6. What are study designs?
Structured approaches to address specific research questions
Provide general guidelines for thinking about specific aspects of study conduct:
sampling populations
systematically collecting measurements
analysing data
Strengths & limitations of specific designs are well-established

7. How to select a study design
Start with a good study question
Relevant
Addresses topic of significance to health of local population / health care services
Novel
Makes meaningful contribution to existing knowledge ~ new insights
Feasible
Not overly ambitious
Creativity

8. Different types of study questions lead to different types of study designs
Descriptive
What is the prevalence of condition Z in a specific population?
Analytic
What are the factors associated condition Z? Is condition X a risk factor for condition Z?
Diagnostic
How good is test Q in detecting condition Z?

9. Selecting the right study design option
Relevant
Design allows you to answer your research question
Novel
Design allows meaningful contribution to existing knowledge ~ new insights
Feasible
Design allows study to be done within available time and funds
Simple
ALWAYS avoid ALL unnecessary complexities

10. Types of study designs Many types
Most are some variation on general themes presented here
All designs based on same basic principles
key differences in how study design samples participants with respect to
“exposures” (risk factors, patient characteristics)
“outcomes” (diseases, conditions)

11. Choice of study design closely related to other aspects of protocol
Study design choices inform how you will sample a specific study population
in a way that its understood how the participants in the study relate to the population in general
2. Study design choices inform the most appropriate measurements to collect on participants in a standardised manner (create data)

12. 3. Study designs will point to the most appropriate analysis of data to answer study question:
Descriptive
Calculate the proportion of the study population with condition Z (incident or prevalent)
Analytic
Compare the frequency of condition Z among groups of the population
Diagnostic
Calculate the validity (sensitivity/specificity) or reliability of test Q in detecting condition Z

13. Broad categories of options in study design
Cross-sectional
Case report / case series
Case-control
Cohort
Randomised Controlled Trial (RCT)

14. Broad categories of options in study design
Cross-sectional
Case series
Case-control
Cohort
Randomised Controlled Trial
Diagnostic ?
Descriptive
Analytic

15. Broad categories of options in study design
Cross-sectional
Case report/series
Case-control
Cohort
RCT
Observational designs:
investigator is only observing distribution of variables (risk factors, diseases, etc) ‘in nature’
Experimental designs:
investigator assigns study conditions ~ usually testing an intervention
(many variations here)

16. Key differences between study designs
How participants are sampled
Are participants sampled according to exposure status, disease status, neither, both?
When measurements are taken
Are some variables measured before others, or are measurements taken all at once?
How outcome variables are measured
Incident or prevalent outcomes (morb/mort)?
Are there comparison groups involved?
Is design observational or experimental?

17. Time marches on
Onset of conditions takes place over time in populations
Different study designs deal with the onset of conditions through time in different ways
Critical to understand how your choice of study design handles the timing of
Identification of participants
Measurement of variables (exposure, disease)

18. X Died Died X Died X X Died X X Died Time X= onset of
condition of interest 1 X
Died 2
Died 3 4 5 X
Died 6 7 X 8 X
Died 9 X 10 X
Died 11
Time

19. II. Details on categories of study designs
Case report & case series
Case-control
Cross-sectional
Cohort
RCT & other experimental designs

20. Case-report & case-series
people with health outcome
depends on what is of interest
Case report / series
Describes
characteristics of disease / condition
characteristics of individual that may be associated with the condition

21. Issues in case-only designs
Useful for descriptive purposes only
Implicit comparisions to what is ‘expected’ or ‘normal’
Why might this be problematic? vs

22. Case-control studies Set of cases (usually from health service)
Comparable set of controls (various sources)
Both groups evaluated on characteristics / ‘exposures’ of interest
Compare distribution of ‘exposure’ in cases and controls

23. Exposed
Cases
Unexposed
Exposed
Controls
Unexposed

24. Exposed
Cases
Unexposed
Exposed
Controls
Unexposed
Time

25. Example: Does children’s inhalation of hairspray facilitates development of asthma?
50 new cases of severe asthma identified at RXH in 12-month period, all <5 yrs of age
These cases are compared to
90 children <5 years attending
RXH for orthopedic surgery
(who do not have asthma)
Cases and controls are
compared on maternal hairspray
use since child’s birth

26. Hairspray
Children
with asthma
No hairspray
Hairspray
Children
without asthma
No hairspray

27. A E- D C B E+ Odds ratio in 2x2 table Odds ratio = (A/C) / (B/D) Cases
Controls

28. Strengths & limitations of case-control study
Relatively simple & quick approach to address analytic questions
Ideal to study rare diseases (vs cohort)
Cases & potential controls are accessible in health care setting
Choice of the wrong control group ~ selection bias
Cases may over-report past exposures ~ information bias

29. Cross-sectional studies
Most common form of research ~ “surveys”
Measure all variables on participants at same point in time (approximately)
Measure prevalent disease (not incidence)
X Disease
X Exposure
Time

30. Collect data on outcome (disease) and exposure (risk factors)
Defined population
Sampling
Collect data on outcome (disease) and exposure (risk factors)
Exposed
Diseased
Not exposed
Diseased
Exposed
Not diseased
Not exposed
Not Diseased

31. Example: How severe is disease among rheumatoid arthritis patients attending GSH?
Study population: patients attending rheumatology clinic at GSH during one month period
Measures: degree of disease severity (outcome); demographics, disease history, treatment history (exposures)
Analysis: prevalence of severe disease in clinic population; association between severity of disease and different exposures

32. Benefits of cross-sectional study
Feasibility → easy to do
In health care setting, can work from existing records (consent issues)
Low cost, rapid
Not waiting for incident outcomes to develop
Can calculate prevalence
Often most relevant measure for burden of disease, informing health care strategies
Measure of association: calculate Odds Ratio for prevalent disease

33. Issues in cross-sectional studies
Measuring prevalent disease only
Prevalence incorporates incidence of disease AND duration of disease
Risk factors for prevalent disease often different from risk factors for incident disease
Issues of timing (temporality) are a problem
Exactly when did disease develop?
Did exposures come before or after onset of disease?

34. Cohort studies
Start with group of individuals without the outcome of interest: “at risk”
Follow forward in time to observe incidence of disease (a rate)
Can be descriptive or analytic
If analytic question, then measure exposures on cohort at the beginning of the study

35. Cohort studies can be purely descriptive Eg: What is the rate of remission among men treated for prostate cancer at GSH?
At risk participants
(without outcome)
Develop outcome of interest
Do not develop outcome of interest
Time

36. Analytic cohort study Eg: Do β-blockers increase risk of renal transplant rejection?
Develop the outcome of interest
Exposure
Study population without the outcome of interest
Do not develop the outcome of interest
Develop the outcome of interest
No exposure
Do not develop outcome of interest
Time

37. Types of cohort studies
Prospective
Following cohort forward through time from present
Most common approach
Retrospective
Assemble cohort from medical records,
“follow” based on records
Follow-up is in the past (can extend into present)

38. RR = [A/(A+B)] / [C/(C+D)]
Measure of association in a cohort study: relative risk (aka risk ratio, rate ratio)
New cases of outcome
Participants who do not develop outcome A B
Exposed
Total number of exposed = A + B C D
Unexposed
Total number of unexposed = C + D
Total number of participants = A + B +C + D
RR = [A/(A+B)] / [C/(C+D)]

39. Strengths + problems in cohort studies
Can calcluate rates of new events~ valuable
Timing of exposure before disease assured
Good for studying health effects of rare exposures (can select an exposed cohort)
Weaknesses
Participants ‘self-select’ their exposure status~ leads to confounding, bias
Take time, resources (if prospective)
Many subjects needed for rare outcomes

40. Randomised controlled trials
Principal experimental design in medical research
Like a cohort study, except exposure status is assigned by investigator (randomly)– not just observed
Complex, take time → costly
RCTs are usually best design for testing the impact of a specific intervention in improving a specific health outcome

41. Randomisation Have outcome of interest Exposure
Study population without the outcome of interest
Do not develop outcome of interest
Randomisation
Have outcome of interest
No exposure
Do not develop outcome of interest
Time

42. Key features of RCT Randomisation Use of concurrent control groups
Removes selection bias or confounding
Alternation or other assignment schemes are bad idea
Use of concurrent control groups
Vs Before/After studies
Blinding whenever possible
Blind investigators: prevents information biases
Blind participants: prevents selective behaviour change during the trial
Not just in trials

43. RCTs are important tools
But can encounter major problems that hinder interpretation of results
Generalizability: Trial participants are highly selected individuals
often not representative of general population at risk
Complexity: Trials procedures can be complex (and costly)
when key design features breakdown, the experiment is compromised

44. Other experimental designs
For an experiment, need to compare 2 states: with intervention vs without
Before/after studies
Introduction of Pfizer
fluconazole donation
programme at GFJ
Median survival of cryptococcal meningitis in HIV+ before
Median survival of cryptococcal meningitis in HIV+ after

45. MMH vs NSH Controlled before/after studies BEFORE AFTER
New training in
sterile procedures
MMH
intervention vs
Rate of postoperative
sepsis
Rate of postoperative
sepsis
No new training
NSH
control
Rate of postoperative
sepsis
Rate of postoperative
sepsis

46. Time-series studies Rate of advanced cervical cancer cases per 100,000
# of pap smears performed in Western Cape

47. III. Conclusion

48. The “hierarchy” of study designs
Frequently see framework for comparing evidence based on the study design used
RCT / experiments
Cohort
Case-control
Cross-sectional
Case series/report
‘better evidence’
~ more valid
‘worse evidence’
~ less valid

49. Not (nearly) so simple
The study design alone does not make the evidence from a study better or worse
The details of how a study is conducted is what matters
Rigour in design, sampling, measurements, analysis
This is why the Methods section is the most important part of scientific papers

50. Wrap-up
Framework for thinking about study designs when developing research ideas for MMed
Start with a good research question
Understand different study design options
Select the most feasible study design based on the study question
Balance time, funding, available data sources
Understand the strengths and limitations of your approach
Be able to justify your choice of study designs

51. Resources to learn more
Consultations re: study design, conduct, analysis
Ask for help before you start collecting data!!
Dr Jim teWaterNaude (to ID appropriate support within School of Public Health)
Self-learning
Hulley SB, Cummings SR. Designing Clinical Research
Gordis L. Epidemiology
Szklo M, Nieto J. Epidemiology: Beyond the Basics
Friedman LM, Fundamentals of Clinical Trials

52. IV. Examples

53. #1
An investigator is interested in studying the association between schizophrenia and measles vaccinations.
Hypothesis: childhood vaccinations predispose individuals to develop schizophrenia in later life
What study designs are possible?
Which design would you recommend and why?

54. #2
A study among outpatients attending the GSH diabetes clinic during 2004 collects data on 1432 patients.
Each patient is included once only in the dataset (ie, info from their first visit during 2004)
Data are collected on patient & family history, past treatment, knowledge of disease & its management, disease severity (GTT)
Medicine Registrar decides to examine whether patients with more severe disease have better knowledge of disease & management
What kind of study of this? What measures can be calculated?

55. #3
You have collected records from your weekly clinic with information on 152 patients– you have data on:
patient demographic characteristics
detailed clinical information on morbidity
medical history
risk behaviours (smoking, drinking)
medications
You need to write an MMed. Quickly.
Identify a research question, a study design, and describe these briefly.
What are the strengths & limitations of the study design you have selected in answering your specific question?

56. #4
A study of neural tube defects and antenatal folate supplementation, lasting 10 years, follows 10,000 pregnancies in which women used folate supplements, and 10,000 pregnancies in which no supplements were used.
Among women taking folate supplements, 50 cases of neural tube defects were observed
Among women not taking supplements, 150 cases of neural tube defects were observed.

57. #4 What type of study is this?
What is the appropriate measure of association?
Draw up a 2x2 table, calculate the measure and interpret in one sentence

58. #5
Another study of folate supplementation and neural tube defects uses a hospital referral system to identify all cases of neural tube defects in the local population.
200 cases are identified over 10 years (50 among women using supplements, 150 among women not using supplements).
For comparison, investigators select 800 control pregnancies (where no neural tube defects were observed) at random from the same population (of whom 498 use folate supplements and 492 are unexposed).

59. #5 What type of study is this?
What is the appropriate measure of association?
Draw up a 2x2 table, calculate the measure and interpret in one sentence.
Comparing #4 and #5, what is the principle advantage of the case-control design to cohort design?