Presentation on theme: "Observational Study Designs and Studies of Medical Tests"— Presentation transcript:
1 Observational Study Designs and Studies of Medical Tests Tom NewmanAugust 17, 2010Thanks to Michael Kohn
2 Outline Conceptual overview Review common observational study designs Cohort, Double CohortCase-ControlCross-sectionalStudies of Medical TestsDiagnostic Test AccuracyPrognostic Test AccuracyExamples“Name that study”
3 Caveats Nomenclature is confusing and used inconsistently “Cross-sectional” can refer to timing or sampling“Retrospective” does not always mean retrospectiveGetting the name right is helpful, but it is more important to be able to explain what you want to do and have it make sense for your RQIf you can’t name your study it’s worth making sure it makes sense
4 Key elements of study design Timing of the studyTiming of variable occurrence and measurementHow the subjects will be sampled
5 Timing of the studyProspective: investigator enrolls subjects and makes measurements in the present and futureHistorical: investigator relates predictor variables that have already been measured to outcomes that have already occurredRetrospective: can mean historical, but best reserved for case-control studies
6 Prospective studiesControl over subject selection and variable measurementsHave to wait for outcomes to occurTake longerMore expensive
7 Historical studiesLess control over subject selection and variable measurementsOutcomes have already occurredDone soonerLess expensive
8 Timing of measurements Longitudinal: measurements in subjects made at more than one timeCross-sectional: predictor and outcome measured at the same time
9 Longitudinal timing of measurements Predictor variable precedes outcomeBetter for causality (reduces likelihood of “effect-cause”)Measurement of predictor precedes measurement of outcomeNo need for blinding of measurement of predictor variableNeeded to measure incidence = new cases/population at risk/timeRisk of getting the disease
10 Cross-sectional timing of measurements Measurement of predictor and outcome at about the same timeCausality may be more difficult to inferNo loss to follow-upCan only measure prevalence = existing cases at one point in time/population at riskPrevalence = incidence x durationRisk of having the diseaseNot as good for causality
11 Example: “Incidence-Prevalence Bias” In asymptomatic adults, prevalence of coronary calcium is lower in blacks than in whites*Does this mean blacks get less heart disease?No, incidence is greater, but duration is shorter***Doherty TM et al J Am Coll Cardiol. 1999;34:787–794**Nieto FJ, Blumenthal RS. J Am Coll Cardiol, 2000; 36:
12 Sampling of subjects Usually best By predictor variable By outcome variableBy other (e.g., demographic) factors that define the population of interestSometimes called “cross-sectional” samplingUsually best
13 Study designs Descriptive Analytical Many studies of medical tests Hint variables must VARYIf either the predictor or outcome variable does not vary in your study (e.g., because one value is an inclusion criterion) your study is descriptiveAnalytical
16 Observational analytic studies Causality is importantMay be only ethical option for studying risk factors for diseaseOften more efficientPopulations may be more representativeMore intellectually interesting than RCTs?
17 Note on FiguresFollowing schematics of observational study designs assume:Predictor = Risk FactorOutcome = DiseaseBoth dichotomous
21 Cohort Studies1) Measure predictor variables on a sample from a population (defined by something other than the variables you are studying).2) Exclude any subjects who already have the outcome.3) Follow the subjects over time and attempt to determine outcome on all subjects.
22 Cohort Studies are longitudinal Can identify individuals lost to follow upCan estimate the incidence of the outcome in the population (e.g., cases/person-year)Measure of disease association is the relative risk (RR) or relative hazard (RH)
24 Double Cohort (Exposed-Unexposed) Studies Sample study subjects separately based on predictor variableExclude potential subjects in whom outcome has already occurred.Attempt to determine outcome in all subjects in both samples over time.
25 Double Cohort (Exposed-Unexposed) Studies Can identify individuals lost to follow upCan measure incidence in each cohort, but not overall incidence in the population*Measure of disease association is the relative risk (RR) or relative hazard (RH)*Unless one of the cohorts is a sample of everyone not in the other cohort
26 Cohort Studies: Summary Timing of the STUDYProspectiveHistoricalTiming of the MEASUREMENTS:All cohort studies are longitudinal (follow patients over time)SAMPLINGCohort study – sample based on other (e.g., demographic) characteristicsDouble cohort study -- sample on predictor variable
28 Case-Control Study1) Separately sample subjects with the outcome (cases) and without the outcome (controls)2) Attempt to determine predictor status on all subjects in both outcome groups
29 Case-Control StudyCannot identify individuals lost to follow up (no such thing as “lost to follow up”, since by definition outcome status is known)Cannot calculate prevalence (or incidence) of outcomeMeasure of disease association is the Odds Ratio (OR)Try to replicate a nested case control study in which the cases and controls arise from the same cohort.
32 Cross-Sectional Study Attempt to determine predictor and outcome status on all patients in a single population (defined by something other than predictor or outcome).
33 Cross-Sectional Study No loss to follow-upCan calculate prevalence but not incidenceMeasure of disease association is the Relative Prevalence (RP).Can be prospective or historical
34 Cohort Studies Start with a Cross-Sectional Study Eliminate subjects who already have disease
35 Studies of Medical Tests Causality often irrelevant.Not enough to show that test result is associated with disease status or outcome*.Need to estimate parameters (e.g., sensitivity and specificity) describing test performance.*Although if it isn’t, you can stop.
36 Studies of Diagnostic Test Accuracy for Prevalent Disease Predictor = Test ResultOutcome = Disease status as determined by Gold StandardDesigns:Case-control (sample separately from disease positive and disease negative groups)Cross-sectional (sample from the whole population of interest)Double-cohort-like sampling (sample separately from test-positive and test-negative groups)
38 Studies of Dx Tests Importance of Sampling Scheme If sampling separately from Disease+ and Disease– groups (case-control sampling), cannot calculate prevalence, positive predictive value, or negative predictive value.
43 Outcomes: (blinded) IQ test and neurologic examination at age 5 years. Babies born at Kaiser with severe neonatal hyperbilirubinemia (Bili 25) were compared with randomly selected “controls” from the same birth cohort.Outcomes: (blinded) IQ test and neurologic examination at age 5 years.Results: No difference in IQ or fraction with neurologic disability between the “case” and “control” groups.Newman, T. B., P. Liljestrand, et al. (2006). N Engl J Med 354(18):
44 Jaundice and Infant Feeding Study Design?(Be Careful)
45 Double Cohort (Exposed-Unexposed) Study* JIFeeDouble Cohort (Exposed-Unexposed) Study*The subjects are divided by predictor (Bili 25+), not outcome (neurologic disability). The “cases” are actually the exposed group and the “controls” are actually the unexposed group*Actually a nested triple cohort study, since “cases” and “controls” came from the same birth cohort and we also studied dehydration. See Hulley page 104.
46 HIV Tropism and Rapid Progression* Is HIV CXCR4 (as opposed to CCR5) tropism a predictor of rapid progression in acutely infected HIV patients?Molecular tropism assay is expensive. Have funding to perform a total of 80 assays.UCSF OPTIONS cohort follows patients acutely infected with HIV. Has banked serum from near time of acute infection.* Vivek Jain’s Project
47 HIV Tropism and Rapid Progression (continued) Identify the 40 patients with the most rapid progression (Group 1) and randomly select 40 others from the UCSF Options cohort (Group 2).Run the tropism assay on banked serum for these 80 patients and compare results between Group 1 and Group 2.
49 HIV Tropism and Rapid Progression Nested Case-Control Study
50 RRISK (Reproductive Risk Factors for Incontinence at Kaiser) Random sample of 2100 women aged years oldInterview, self report, diaries to determine whether they have the outcome, urinary incontinence.Chart abstraction of obstetrical and surgical records to establish predictor status
52 RRISK Funded with an R01 by the NIDDK as a retrospective cohort study Longitudinal, but can’t tell loss to follow-up, incidence of incontinence, or relative risk from this designMichael calls it a cross-sectional studyTells us prevalence of incontinenceBut not all measurements made at the same timeIt’s a lot like a nested case control studyBut did not employ “case-control” samplingNested cross-sectional study?
53 Steroid treatment in the ED and among children hospitalized for asthma Research Question: what are the frequency and predictors of delayed receipt of steroids in the ED among children admitted for asthma?Subjects: children admitted for asthmaPredictors: age, time of arrival, etc.Outcome: Time to steroid administrationwhat are the frequency and predictors of delayed receipt of steroids in the ED among children admitted for asthma?
54 Steroid treatment in the ED and among children hospitalized for asthma -2 This study is hard to nameTime to event data makes this sound like a cohort study (even if follow-up time is very short)Define a group at risk of the outcomeMeasure predictorsFollow for outcome occurrenceBut there is a problem:You can’t define a cohort based on variables not present at baseline
55 Steroid treatment in the ED and among children hospitalized for asthma -3 Possible changesMake it a descriptive study of hospitalized patientsMake it a cohort study of ED patientsCould study predictors of time to steroidsTime to steroids could be a predictor of hospitalization
56 Association of lipid‑laden alveolar macrophages (LLAM) and gastroesophageal reflux (GER) in children*Did pH probe, barium swallow, and endoscopy on 115 children with chronic respiratory tract disorders to determine GE refluxGroup 1: 74 children with GERGroup 2: 41 children with no GERBronchoscopy and bronchial lavage to determine LLAMLLAM were present in 63/74 (85%) with GERLLAM in 8/41 (19%) children without GERP <Design?*J Pediatr 1987;110:190‑4
57 Association of lipid‑laden alveolar macrophages (LLAM) and gastroesophageal reflux (GER) in children*Design: Cross-sectional study of diagnostic test accuracy (with cross-sectional sampling)*J Pediatr 1987;110:190‑4
58 Association of lipid‑laden alveolar macrophages and gastroesophageal reflux in children -3 Conclusions: “We suggest that LLAM from bronchial lavage may be a useful marker for tracheal aspiration in children with GER in whom chronic lung disease may subsequently develop.”What is wrong?J Pediatr 1987;110:190‑4
59 Association of lipid‑laden alveolar macrophages and gastroesophageal reflux in children -4 Conclusions: “We suggest that LLAM from bronchial lavage may be a useful marker for tracheal aspiration in children with GER in whom chronic lung disease may subsequently develop.”Study design does not permit this conclusionCan’t estimate risk of developing lung disease from aCross-sectional sample thatIncludes only patients with lung diseaseJ Pediatr 1987;110:190‑4