2. Analysis of matched data; plus, diagnostic testing

3. Correlated Observations Correlated data arise when pairs or clusters of observations are related and thus are more similar to each other than to other observations in the dataset. Ignoring correlations will: – overestimate p-values for within-person or within-cluster comparisons – underestimate p-values for between-person or between-cluster comparisons

4. Pair Matching: Why match? Pairing can control for extraneous sources of variability and increase the power of a statistical test. Match 1 control to 1 case based on potential confounders, such as age, gender, and smoking.

5. Example Johnson and Johnson (NEJM 287: 1122-1125, 1972) selected 85 Hodgkin’s patients who had a sibling of the same sex who was free of the disease and whose age was within 5 years of the patient’s…they presented the data as…. Hodgkin’s Sib control TonsillectomyNone 4144 33 52 From John A. Rice, “Mathematical Statistics and Data Analysis. OR=1.47; chi-square=1.53 (NS)

6. Example But several letters to the editor pointed out that those investigators had made an error by ignoring the pairings. These are not independent samples because the sibs are paired…better to analyze data like this: From John A. Rice, “Mathematical Statistics and Data Analysis. OR=2.14*; chi-square=2.91 (p=.09) Tonsillectomy None TonsillectomyNone 2615 7 37 Case Control

7. Pair Matching: example Match each MI case to an MI control based on age and gender. Ask about history of diabetes to find out if diabetes increases your risk for MI.

8. Pair Matching: example Which cells are informative? Just the discordant cells are informative! Diabetes No diabetes 25119 DiabetesNo Diabetes 937 16 82 46 98 144 MI cases MI controls

9. Pair Matching Diabetes No diabetes 25119 DiabetesNo Diabetes 937 16 82 46 98 144 MI cases MI controls OR estimate comes only from discordant pairs! The question is: among the discordant pairs, what proportion are discordant in the direction of the case vs. the direction of the control. If more discordant pairs “favor” the case, this indicates OR>1.

10. Diabetes No diabetes 25119 DiabetesNo Diabetes 937 16 82 46 98 144 MI cases MI controls P(“favors” case/discordant pair) =

11. Diabetes No diabetes 25119 DiabetesNo Diabetes 937 16 82 46 98 144 MI cases MI controls odds(“favors” case/discordant pair) =

12. Diabetes No diabetes 25119 DiabetesNo Diabetes 937 16 82 46 98 144 MI cases MI controls OR estimate comes only from discordant pairs!! OR= 37/16 = 2.31 Makes Sense!

13. Diabetes No diabetes DiabetesNo Diabetes 937 16 82 MI cases MI controls McNemar’s Test Null hypothesis: P(“favors” case / discordant pair) =.5 (note: equivalent to OR=1.0 or cell b=cell c)

14. Diabetes No diabetes DiabetesNo Diabetes 937 16 82 MI cases MI controls McNemar’s Test Null hypothesis: P(“favors” case / discordant pair) =.5 (note: equivalent to OR=1.0 or cell b=cell c) By normal approximation to binomial:

15. McNemar’s Test: generally By normal approximation to binomial: Equivalently: exp No exp expNo exp ab c d cases controls

16. Diabetes No diabetes DiabetesNo Diabetes 937 16 82 MI cases MI controls McNemar’s Test McNemar’s Test:

17. Example: McNemar’s EXACT test Split-face trial: – Researchers assigned 56 subjects to apply SPF 85 sunscreen to one side of their faces and SPF 50 to the other prior to engaging in 5 hours of outdoor sports during mid-day. The outcome is sunburn (yes/no). – Unit of observation = side of a face – Are the observations correlated? Yes. Russak JE et al. JAAD 2010; 62: 348-349.

18. Results ignoring correlation: Table I -- Dermatologist grading of sunburn after an average of 5 hours of skiing/snowboarding (P =.03; Fisher’s exact test) Sun protection factorSunburnedNot sunburned 85155 50848 Fisher’s exact test compares the following proportions: 1/56 versus 8/56. Note that individuals are being counted twice!

19. Correct analysis of data: Table 1. Correct presentation of the data (P =.016; McNemar’s exact test). SPF-50 side SPF-85 sideSunburnedNot sunburned Sunburned10 Not sunburned748 McNemar’s exact test: Null hypothesis: X~binomial (n=7, p=.5)

20. Standard error of the difference of two proportions= RECALL: 95% confidence interval for a difference in INDEPENDENT proportions Standard error can be estimated by:95% confidence interval for the difference between two proportions:

21. 95% CI for difference in dependent proportions Variance of the difference of two random variables is the sum of their variances minus 2*covariance:

22. 95% CI for difference in dependent proportions Diabetes No diabetes 25119 DiabetesNo Diabetes 937 16 82 46 98 144 MI cases MI controls

23. The connection between McNemar and Cochran-Mantel-Haenszel Tests

24. View each pair is it’s own “age-gender” stratum Diabetes No diabetes Case (MI)Control 11 0 0 Example: Concordant for exposure (cell “a” from before)

25. Diabetes No diabetes Case (MI)Control 11 0 0 Diabetes No diabetes Case (MI)Control 10 0 1 x 9 x 37 Diabetes No diabetes Case (MI)Control 01 1 0 Diabetes No diabetes Case (MI)Control 00 1 1 x 16 x 82

26. Mantel-Haenszel for pair- matched data We want to know the relationship between diabetes and MI controlling for age and gender (the matching variables). Mantel-Haenszel methods apply.

27. RECALL: The Mantel-Haenszel Summary Odds Ratio Exposed Not Exposed CaseControl ab c d

28. Diabetes No diabetes Case (MI)Control 11 0 0 Diabetes No diabetes Case (MI)Control 10 0 1 ad/T = 0 bc/T=0 ad/T=1/2 bc/T=0 Diabetes No diabetes Case (MI)Control 01 1 0 Diabetes No diabetes Case (MI)Control 00 1 1 ad/T=0 bc/T=1/2 ad/T=0 bc/T=0 x 9 x 37 x 16 x 82

29. Mantel-Haenszel Summary OR

30. Mantel-Haenszel Test Statistic (same as McNemar’s)

31. Concordant cells contribute nothing to Mantel- Haenszel statistic (observed=expected) Diabetes No diabetes Case (MI)Control 11 0 0 Diabetes No diabetes Case (MI)Control 00 1 1

32. Discordant cells Diabetes No diabetes Case (MI)Control 10 0 1 Diabetes No diabetes Case (MI)Control 01 1 0

35. From: “Large outbreak of Salmonella enterica serotype paratyphi B infection caused by a goats' milk cheese, France, 1993: a case finding and epidemiological study” BMJ 312: 91- 94; Jan 1996. Example: Salmonella Outbreak in France, 1996

37. Epidemic Curve

38. Matched Case Control Study Case = Salmonella gastroenteritis. Community controls (1:1) matched for:  age group ( = 65 years)  gender  city of residence

39. Results

40. In 2x2 table form: any goat’s cheese Goat’s cheese None 2930 Goat’ cheeseNone 23 6 7 46 13 59 Cases Controls

41. In 2x2 table form: Brand A Goat’s cheese Goat’s cheese B None 1049 Goat’ cheese BNone 824 2 25 32 27 59 Cases Controls

42. Brand A None Case (MI)Control 11 0 0 Brand A None Case (MI)Control 10 0 1 Brand A None Case (MI)Control 01 1 0 Brand A None Case (MI)Control 00 1 1 x8x8 x 24 x2x2 x 25

43. Summary: 8 concordant-exposed pairs (=strata) contribute nothing to the numerator (observed-expected=0) and nothing to the denominator (variance=0). Summary: 25 concordant-unexposed pairs contribute nothing to the numerator (observed-expected=0) and nothing to the denominator (variance=0). Using Agresti notation here!

44. Summary: 2 discordant “control-exposed” pairs contribute -.5 each to the numerator (observed-expected= -.5) and.25 each to the denominator (variance=.25). Summary: 24 discordant “case-exposed” pairs contribute +.5 each to the numerator (observed-expected= +.5) and.25 each to the denominator (variance=.25).

46. Diagnostic Testing and Screening Tests

47. Characteristics of a diagnostic test Sensitivity= Probability that, if you truly have the disease, the diagnostic test will catch it. Specificity=Probability that, if you truly do not have the disease, the test will register negative.

48. Calculating sensitivity and specificity from a 2x2 table +- +ab -cd Screening Test Truly have disease Sensitivity Specificity Among those with true disease, how many test positive? Among those without the disease, how many test negative? a+b c+d

49. Hypothetical Example +- +91 -109881 Mammography Breast cancer ( on biopsy) Sensitivity=9/10=.90 10 990 Specificity= 881/990 =.89 1 false negatives out of 10 cases 109 false positives out of 990

50. What factors determine the effectiveness of screening? The prevalence (risk) of disease. The effectiveness of screening in preventing illness or death. – Is the test any good at detecting disease/precursor (sensitivity of the test)? – Is the test detecting a clinically relevant condition? – Is there anything we can do if disease (or pre-disease) is detected (cures, treatments)? – Does detecting and treating disease at an earlier stage really result in a better outcome? The risks of screening, such as false positives and radiation.

51. Positive predictive value The probability that if you test positive for the disease, you actually have the disease. Depends on the characteristics of the test (sensitivity, specificity) and the prevalence of disease.

52. Example: Mammography Mammography utilizes ionizing radiation to image breast tissue. The examination is performed by compressing the breast firmly between a plastic plate and an x-ray cassette that contains special x-ray film. Mammography can identify breast cancers too small to detect on physical examination. Early detection and treatment of breast cancer (before metastasis) can improve a woman ’ s chances of survival. Studies show that, among 50-69 year-old women, screening results in 20-35% reductions in mortality from breast cancer.

53. Mammography Controversy exists over the efficacy of mammography in reducing mortality from breast cancer in 40-49 year old women. Mammography has a high rate of false positive tests that cause anxiety and necessitate further costly diagnostic procedures. Mammography exposes a woman to some radiation, which may slightly increase the risk of mutations in breast tissue.

54. Example A 60-year old woman has an abnormal mammogram; what is the chance that she has breast cancer? E.g., what is the positive predictive value?

55. Calculating PPV and NPV from a 2x2 table +- +ab -cd Screening Test Truly have disease PPV NPV Among those who test positive, how many truly have the disease? Among those who test negative, how many truly do not have the disease? a+cb+d

56. Hypothetical Example +- +91 -109881 Mammography Breast cancer ( on biopsy) PPV=9/118=7.6% 118882 Prevalence of disease = 10/1000 =1% NPV=881/882=99.9%

57. What if disease was twice as prevalent in the population? +- +182 -108872 Mammography Breast cancer ( on biopsy) sensitivity=18/20=.90 20 980 specificity=872/980=.89 Sensitivity and specificity are characteristics of the test, so they don’t change!

58. What if disease was more prevalent? PPV=18/126=14.3% 126874 Prevalence of disease = 20/1000 =2% NPV=872/874=99.8% +- +182 -108872 Mammography Breast cancer ( on biopsy)

59. Conclusions Positive predictive value increases with increasing prevalence of disease Or if you change the diagnostic tests to improve their accuracy.