1. Evidence-Based Medicine 3 More Knowledge and Skills for Critical Reading Karen E. Schetzina, MD, MPH

2. Epidemiology Definition - the study of the distribution and determinants of health-related states or events in specified populations, and the application of this study to the control of health problems. Definition - the study of the distribution and determinants of health-related states or events in specified populations, and the application of this study to the control of health problems. Epidemiologists and clinical researchers study samples of populations, collect information on variables of interest from persons in the samples, and then look for associations between the variables of interest. Epidemiologists and clinical researchers study samples of populations, collect information on variables of interest from persons in the samples, and then look for associations between the variables of interest. Through this process, hypotheses are generated, causes of disease are identified, treatments are discovered, etc. Through this process, hypotheses are generated, causes of disease are identified, treatments are discovered, etc.

3. The Effect of Race and Sex on Physicians’ Recommendations for Cardiac Catheterization The Effect of Race and Sex on Physicians’ Recommendations for Cardiac Catheterization Design: Computerized Survey Design: Computerized Survey Surveyed 720 primary care physicians attending two national meetings Surveyed 720 primary care physicians attending two national meetings Physicians viewed video recorded interviews of black and white male and female patients ages 55 – 70 with chest pain, as well as results of their electrocardiography and thallium stress tests Physicians viewed video recorded interviews of black and white male and female patients ages 55 – 70 with chest pain, as well as results of their electrocardiography and thallium stress tests The physicians then were asked whether they wished to refer the patient for cardiac catheterization The physicians then were asked whether they wished to refer the patient for cardiac catheterization Critical Reading - Review

4. What is a variable? What is a variable? What were the main predictor variables in this study? What were the main predictor variables in this study? What was the main outcome variable in this study? What was the main outcome variable in this study?

5. Critical Reading – Review Results – Table 4: Referral for Cardiac Catheterization According to Experimental Factor Results – Table 4: Referral for Cardiac Catheterization According to Experimental Factor Factor Mean Referral Rate (%) Odds Ratio (95% CI) P Value MaleFemale90.684.51.0 0.6 (0.4-0.9) 0.02 WhiteBlack90.684.71.0 0.02

6. Critical Reading – Review Odds Ratio – The ratio of two odds. For rare diseases, this approximates relative risk. Commonly calculated in cross-sectional studies and case control studies, and from logistic regression. Odds Ratio – The ratio of two odds. For rare diseases, this approximates relative risk. Commonly calculated in cross-sectional studies and case control studies, and from logistic regression. Interpretation: Interpretation: >1 suggests positive association >1 suggests positive association <1 suggests negative association <1 suggests negative association =1 suggests no difference between groups =1 suggests no difference between groups

7. Odds Ratio – Cardiac Catheterization Article Referred Not Referred Referred Not Referred Female Female Male Male OR =odds of referral for females = A/B=AD OR =odds of referral for females = A/B=AD odds of referral for males = C/D BC odds of referral for males = C/D BC

8. Odds Ratio – General Definition D+ D- D+ D- E+ E+ E- E- OR =odds of disease for E+ = A/B=AD OR =odds of disease for E+ = A/B=AD odds of disease for E- = C/D BC odds of disease for E- = C/D BC

9. Exposure Odds Ratio – Case Control Study D+ D- D+ D- E+ E+ E- E- OR =odds of exposure for D+ = A/C=AD OR =odds of exposure for D+ = A/C=AD odds of exposure for D- = B/D BC odds of exposure for D- = B/D BC

10. Relative Risk D+ D- D+ D- E+ E+ E- E- RR = Risk of disease for E+ = A/(A + B) RR = Risk of disease for E+ = A/(A + B) Risk of disease for E- C/(C + D) Risk of disease for E- C/(C + D)

11. Absolute Risk D+ D- D+ D- E+ E+ E- E- AR = (Risk for E+) - (Risk for E-) = AR = (Risk for E+) - (Risk for E-) = A/(A + B) - C/(C + D) A/(A + B) - C/(C + D)

12. Critical Reading - Review “In univariate analysis, the race and sex of the patient were significantly associated with the physicians’ decisions about whether to make referrals for cardiac catheterization, with men and whites more likely to be referred than women and blacks, respectively.” “In univariate analysis, the race and sex of the patient were significantly associated with the physicians’ decisions about whether to make referrals for cardiac catheterization, with men and whites more likely to be referred than women and blacks, respectively.”

13. Critical Reading - Review Results – Table 4: Referral for Cardiac Catheterization According to Experimental Factor Results – Table 4: Referral for Cardiac Catheterization According to Experimental Factor Factor Mean Referral Rate (%) Odds Ratio (95% CI) P Value MaleFemale90.684.51.0 0.6 (0.4-0.9) 0.02 WhiteBlack90.684.71.0 0.02

14. Hypothesis Testing Random sampling error exists in all epidemiological studies. Hypothesis testing allows us to account for this random error and to determine whether a result is “statistically significant.” Random sampling error exists in all epidemiological studies. Hypothesis testing allows us to account for this random error and to determine whether a result is “statistically significant.” Hypothesis Testing – Statistically test the study hypothesis against the null hypothesis (the null hypothesis is the nothing hypothesis - says there is no association between two variables – i.e. between risk factor and disease). Hypothesis Testing – Statistically test the study hypothesis against the null hypothesis (the null hypothesis is the nothing hypothesis - says there is no association between two variables – i.e. between risk factor and disease). Study Hypothesis – i.e. - There is an association between sex & race and physicians’ recommendations for cardiac catheterization. Study Hypothesis – i.e. - There is an association between sex & race and physicians’ recommendations for cardiac catheterization.

15. p-Value Test statistic – A value quantifying the degree of association between two variables that is calculated from the statistical test procedure. For example, a chi-square statistic. Test statistic – A value quantifying the degree of association between two variables that is calculated from the statistical test procedure. For example, a chi-square statistic. p-Value - The probability of obtaining a value for the test statistic as extreme or more extreme as that observed if the null hypothesis were true (also calculated from the statistical test procedure). A p-Value quantifies the degree of random variability in the sampling process. p-Value - The probability of obtaining a value for the test statistic as extreme or more extreme as that observed if the null hypothesis were true (also calculated from the statistical test procedure). A p-Value quantifies the degree of random variability in the sampling process.

16. p-Value Statistical Significance – Most researchers are willing to declare that a relationship is statistically significant if the chances of observing the relationship in the sample when nothing is going on in the population are less than 5%. This is why the commonly accepted cut point for calling a result “statistically significant is p<0.05. Statistical Significance – Most researchers are willing to declare that a relationship is statistically significant if the chances of observing the relationship in the sample when nothing is going on in the population are less than 5%. This is why the commonly accepted cut point for calling a result “statistically significant is p<0.05.

17. Confidence Intervals Another value that can be calculated from statistical test procedures that accounts for random sampling error. Another value that can be calculated from statistical test procedures that accounts for random sampling error. 95% Confidence Intervals (95% CI) are commonly reported. 95% Confidence Intervals (95% CI) are commonly reported. 95% CI – A range of values computed from the sample that should contain the true population parameter with 95% probability in repeated collections of the data (i.e. a range of values that is almost sure to contain the true population parameter). 95% CI – A range of values computed from the sample that should contain the true population parameter with 95% probability in repeated collections of the data (i.e. a range of values that is almost sure to contain the true population parameter).

18. Confidence Intervals The width of a confidence interval is inversely proportionate to the sample size of the study. The width of a confidence interval is inversely proportionate to the sample size of the study. For risk ratios and odds ratios, if the confidence interval includes the value “1,” the association is not “statistically significant.” For risk ratios and odds ratios, if the confidence interval includes the value “1,” the association is not “statistically significant.” If the confidence intervals for measures in two groups overlaps, the two groups do not differ “significantly” with respect to that measure. If the confidence intervals for measures in two groups overlaps, the two groups do not differ “significantly” with respect to that measure.

19. Important! p-Values and Confidence Intervals assume that there is no bias, or systematic error, in the study - i.e., they do not account for bias in the study. They do not assure that the association is real. They do not quantify clinical significance. It is important not to completely discount values that are not statistically significant. One must also look at trends and how the results compare to previous studies. p-Values and Confidence Intervals assume that there is no bias, or systematic error, in the study - i.e., they do not account for bias in the study. They do not assure that the association is real. They do not quantify clinical significance. It is important not to completely discount values that are not statistically significant. One must also look at trends and how the results compare to previous studies.

20. Hill’s Causal Criteria Strength Strength Consistency Consistency Specificity Specificity Temporality Temporality Biologic gradient Biologic gradient Plausibility Plausibility Coherence Coherence Experimental evidence Analogy

21. Test Your Knowledge From Table 3 in “Factors Associated with Hypertension Control in the General Population of the United States” From Table 3 in “Factors Associated with Hypertension Control in the General Population of the United States” Age- and sex- adjusted odds ratios and 95% confidence intervals for the association between hypertension control and having private health insurance (compared to no insurance): Age- and sex- adjusted odds ratios and 95% confidence intervals for the association between hypertension control and having private health insurance (compared to no insurance): NHW: 1.64 (0.99-2.70) NHW: 1.64 (0.99-2.70) NHB: 2.62 (1.62-4.26) NHB: 2.62 (1.62-4.26) MA: 1.16 (0.52-2.60) MA: 1.16 (0.52-2.60)

22. Test Your Knowledge From Table 4 in “Factors Associated with Hypertension Control in the General Population of the United States” From Table 4 in “Factors Associated with Hypertension Control in the General Population of the United States” Multivariate Adjusted Odds Ratio and 95% Confidence Intervals and p-Values for the association between hypertension control and marital status: Multivariate Adjusted Odds Ratio and 95% Confidence Intervals and p-Values for the association between hypertension control and marital status: Currently married (compared to never married): Currently married (compared to never married): OR=2.39 (1.52-3.71) p-Value<0.001 OR=2.39 (1.52-3.71) p-Value<0.001

23. Next Lecture We will discuss sources of systematic error (bias) and confounding. Some examples are: We will discuss sources of systematic error (bias) and confounding. Some examples are: 1. Selection-bias (people who volunteer for studies may be different, "healthy-worker effect"). From the study: "Physicians who attend professional meeting may be better informed than those who do not attend... may have a greater interest than others in coronary heart disease." How might findings differ if they sampled all practicing physicians? 1. Selection-bias (people who volunteer for studies may be different, "healthy-worker effect"). From the study: "Physicians who attend professional meeting may be better informed than those who do not attend... may have a greater interest than others in coronary heart disease." How might findings differ if they sampled all practicing physicians? 2. Non-response bias (How do respondents and non- respondents differ in regard to the study question?). This study does not give response rates - only says that 720 physicians participated - at least they did not know that it was a study of the effects of race and sex. 2. Non-response bias (How do respondents and non- respondents differ in regard to the study question?). This study does not give response rates - only says that 720 physicians participated - at least they did not know that it was a study of the effects of race and sex. 3. Measurement bias (How accurately were the predictor and outcome variables measured?) 3. Measurement bias (How accurately were the predictor and outcome variables measured?)

24. Next Lecture Confounding may be considered "a confusion of effects" - attributing a result or disease to a specific risk factor when it is in fact due to another factor It can lead to over- or under-estimation of an effect or can even change the direction of the effect. Confounding may be considered "a confusion of effects" - attributing a result or disease to a specific risk factor when it is in fact due to another factor It can lead to over- or under-estimation of an effect or can even change the direction of the effect. Researchers may attempt to control confounding in several difference ways. From the study: these authors reported that they clothed patients identically and listed them as having the same type of insurance and occupations to help to remove the potential confounding effects of SES and insurance. Researchers may attempt to control confounding in several difference ways. From the study: these authors reported that they clothed patients identically and listed them as having the same type of insurance and occupations to help to remove the potential confounding effects of SES and insurance. Another way the authors attempted to control for confounding was by using a "multivariate logistic regression analysis." From the study: Are the differences between rates of referral by race and sex due to other factors besides just race and sex? Physicians are aware of many different risk factors for coronary heart disease (several are reported in the other article you read). Perhaps they referred the white males in the study more often because they thought that they were at higher risk for coronary artery disease based on their clinical presentation? Well, the authors attempted to account for this possibility in the analysis as well as for other factors (age, level of risk, type of chest pain, results of thallium test) in an attempt to determine the differences in referral rates just based on sex and race as independent factors. Another way the authors attempted to control for confounding was by using a "multivariate logistic regression analysis." From the study: Are the differences between rates of referral by race and sex due to other factors besides just race and sex? Physicians are aware of many different risk factors for coronary heart disease (several are reported in the other article you read). Perhaps they referred the white males in the study more often because they thought that they were at higher risk for coronary artery disease based on their clinical presentation? Well, the authors attempted to account for this possibility in the analysis as well as for other factors (age, level of risk, type of chest pain, results of thallium test) in an attempt to determine the differences in referral rates just based on sex and race as independent factors.