Interpreting Epidemiologic Results

Interpreting Results DECISION H0 True (No assoc.) H1 True (Yes assoc.) Four possible outcomes of any epidemiologic study: DECISION H0 True (No assoc.) H1 True (Yes assoc.) Do not reject H0 (not stat. sig.) Correct decision Type II (beta error) Reject H0 (stat. sig.) Type I (alpha error)

When evaluating the incidence of disease Interpreting Results When evaluating the incidence of disease between the exposed and non-exposed groups, we need guidelines to help determine whether there is a true difference between the two groups, or perhaps just random variation from the study sample.

Interpreting Results “Conventional” Guidelines: • Set the fixed alpha level (Type I error) to 0.05 This means, if the null hypothesis is true, the probability of incorrectly rejecting it is 5% of less. The “p-value” is a measure of the compatibility of the data and the null hypothesis. DECISION H0 True H1 True Do not reject H0 (not stat. sig.) Reject H0 (stat. sig.) Type I (alpha error)

Interpreting Results Example: IE+ = 15 / (15 + 85) = 0.15 D+ D- E+ 15 85 E- 10 90 IE+ = 15 / (15 + 85) = 0.15 IE- = 10 / (10 + 90) = 0.10 RR = IE+/IE- = 1.5, p = 0.30 Although it appears that the incidence of disease may be higher in the exposed than in the non-exposed (RR=1.5), the p-value of 0.30 exceeds the fixed alpha level of 0.05. This means that the observed data are relatively compatible with the null hypothesis. Thus, we do not reject H0 in favor of H1 (alternative hypothesis).

Interpreting Results Conventional Guidelines: • Set the fixed beta level (Type II error) to 0.20 This means, if the null hypothesis is false, the probability of not rejecting it is 20% of less. The “power” of a study is (1 – beta). This means having 80% probability to reject H0 when H1 is true. DECISION H0 True H1 True Do not reject H0 (not stat. sig.) Type II (beta error) Reject H0 (stat. sig.)

Interpreting Results Example: N Incid. E- 200 0.10 E+ 0.18 0.21 0.24 RR 1.8 2.1 2.4 Power 58% 82% 95% With the above sample size of 400, and if the alternative hypothesis is true, we need to expect a RR of about 2.1 (power = 82%) or higher to be able to reject the null hypothesis in favor of the alternative hypothesis.

Interpreting Results Factors that affect the power of a study: 1. The fixed alpha level (the lower the level, the the lower the power). 2. The total and within group sample sizes (the smaller the sample size, the lower the power -- unbalanced groups have lower power than balanced groups). 3. The anticipated effect size (the higher the expected/observed effect size, the higher the power).

Interpreting Results Trade-offs between fixed alpha and beta levels: Reducing the fixed alpha level (e.g. to < 0.01) is considered “conservative.” This reduces the likelihood of a type I error (erroneously rejecting the null hypothesis), but at the expense of increasing the probability of a type II error if the alternative hypothesis is true.

Interpreting Results Trade-offs between fixed alpha and beta levels: Increasing the fixed alpha level (e.g. to < 0.10) reduces the probability of a type II error (failing to reject H0 when H1 is true), but at the expense of increasing the probability of a type I error if the null hypothesis is true.

Interpreting Results Expos-ure N Incid. Risk ratio P- value Power * RR ** None 1000 0.10 1.0 --- Low 500 0.15 1.5 0.006 77% 1.52 Medium 250 0.02 60% 1.64 High 100 0.12 27% 2.08 *Power with given sample size and risk ratio ( = 0.05) **Risk ratio needed for 80% power with given sample size

It is an arbitrary quantity with no direct relationship to biology Interpreting Results The p-value is NOT the index of causality It is an arbitrary quantity with no direct relationship to biology

Interpreting Results Since science is based on measurement, the analysis of epidemiologic data is more of a measurement problem than a problem in decision making (e.g. whether or not p < 0.05). This leads to a range of possible values (“confidence interval”) around the point estimate).

Interpreting Results Confidence Interval: Range of values for a point estimate that has a specified probability of including the true value of the parameter. Confidence Level: (1.0 – ), usually expressed as a percentage (e.g. 95%). Confidence Limits: The upper and lower end points of the confidence interval.

Interpreting Results Example: IE+ = 15 / (15 + 85) = 0.15 D+ D- E+ 15 85 E- 10 90 IE+ = 15 / (15 + 85) = 0.15 IE- = 10 / (10 + 90) = 0.10 RR = IE+/IE- = 1.5, p = 0.30 95% C.I. (0.71, 3.07) Our best estimate is that the risk of disease is 1.5 times higher in persons exposed compared to persons not exposed, however, we are 95% confident that the true value lies somewhere between 0.71 and 3.07.

Interpreting Results lower limit = (0.73)e-0.44 = 0.47 Example Calculation of 95% C.I. For Odds Ratio: D+ D- E+ 32 825 E- 56 1,048 32 / 56 OR = ------------ = 0.73 825 / 1,048 95% C.I. = (OR)exp [(+ 1.96(sqrt(1/a + 1/b + 1/c + 1/d))] = (0.73)exp [+ 1.96 (sqrt(1/32 + 1/825 + 1/56 + 1/1,048))] = (0.73)exp [+ 1.96 (sqrt(0.0513))] =(0.73)e+0.44 lower limit = (0.73)e-0.44 = 0.47 upper limit = (0.73)e+0.44 = 1.13

Interpreting Results Interpretation of C.I. For OR and RR: 32 / 56 825 E- 56 1,048 32 / 56 OR = ------------ = 0.73 825 / 1,048 95% C.I. = 0.47, 1.13 If the 95% confidence interval includes the null value of 1.0, then the test result is not “statistically significant.” Because the OR and RR have a lower bound of 0, but no upper bound, the 95% C.I. will tend to be skewed to the right of the point estimate.

Interpreting Results Interpretation of C.I. For OR and RR: The C.I. provides an idea of the likely magnitude of the effect and the random variability of the point estimate. On the other hand, the p-value reveals nothing about the magnitude of the effect or the random variability of the point estimate. In general, smaller sample sizes have larger C.I.’s due to uncertainty (lack of precision) in the point estimate.

Analytic Study Designs

Analytic Study Designs • Randomized trial • Prospective & retrospective cohort studies • Case-control study • Case-crossover study • Cross-sectional study (prevalence survey)

Analytic Study Designs Randomized trial: • Used to evaluate new approaches to treatment and prevention • Can be conducted in clinical settings (individual subjects), or in the community (groups of subjects) • Investigator (experimenter) selects (assigns) the exposure status of each subject or group (e.g. therapeutic regimen)

Analytic Study Designs Randomized trial: • Subjects/groups are assigned at random to the exposure • Participants are followed and the occurrence of disease is compared between the exposure groups (e.g. risk ratio or rate ratio) • Due to randomization (comparability), may provide the most reliable evidence

Analytic Study Designs Prospective cohort (“follow-up”) study: • Disease free individuals are selected and their exposure status is ascertained • Subjects are followed for a period of time to record and compare the incidence of disease between exposed and non-exposed individuals (e.g. risk ratio or rate ratio) • Good for examining effect of rare exposures and for establishing temporal relationship between exposure and disease

Analytic Study Designs Prospective cohort (“follow-up”) study: Exposure Disease ? ? Exposure may or may not have occurred at study entry Outcome definitely has not occurred at study entry

Analytic Study Designs Retrospective cohort study: • Investigator has access to exposure data on a group of people • The study sample is divided into exposed and non-exposed groups • Both the exposures and outcomes of interest have already occurred (hence “retrospective”) • The disease experience of exposed and non- exposed groups is compared (e.g. risk ratio or rate ratio)

Analytic Study Designs Retrospective cohort study: Exposure Disease ? ? Both exposure and disease have already occurred

Analytic Study Designs Case-control study: • Subjects selected on basis of their disease status (hence good for rare diseases) • An appropriate group of “control” subjects (comparison group) is selected • Both the exposures and outcomes of interest have already occurred • The presence of the exposure is compared between the two groups (e.g. odds ratio)

Analytic Study Designs Case-crossover study: • Only subjects (cases) who have experienced the disease of interest are selected • Investigator postulates a critical exposure period (“empirical induction period”) for the exposure of interest • Presence of the exposure is compared between the critical exposure period and other periods of exposure (e.g. conditional odds ratio) • Good for studying effects of transient exposures

Analytic Study Designs irrelevant (non-causal) Case-crossover study: (A) (B) Hypothesized irrelevant (non-causal) exposure Hypothesized relevant (causal) exposure Outcome event “Empirical induction period” Compare presence of exposure between hypothesized non-causal (A) and hypothesized causal (B) periods of exposure

Case-Crossover Study Hypothesis: Heavy physical insertion (HPI) is associated with short-term (within 48 hours) risk of myocardial infarction Person HPI: 49-96 hours prior to MI HPI: 0-48 hours prior to MI 1 No 2 Yes 3 4 5 6 7 8 How do we calculate the conditional odds ratio (COR)?

Case-Crossover Study Hypothesis: Heavy physical insertion (HPI) is associated with short-term (within 48 hours) risk of myocardial infarction Person HPI: 49-96 hours prior to MI HPI: 0-48 hours prior to MI 1 No 2 Yes 3 4 5 6 7 8 Use only the discordant pairs COR = HPI: 0-48 ------------- HPI: 49-96 = 3 / 2 = 1.5

Cross-Sectional Study • Both a descriptive and analytic study design • Snapshot of the health status of populations at a certain point in time • For each subject, exposure and disease outcome are assessed simultaneously (hence also called a “prevalence study/survey”) • Compare prevalence of disease in persons with and without the exposure of interest (e.g. prevalence ratio – same formula as risk ratio)