Dichotomous Tests Thomas B. Newman, MD, MPH September 27, 2012 Thanks to Josh Galanter and Michael Shlipak 1

Overview n Clarifications, chapter 1, chapter 2 material n Definitions: sensitivity, specificity, prior and posterior probability, predictive value, accuracy n 2 x 2 table method n Likelihood ratios - WOWO n Probability and odds n FP/FN confusion n Test/treat thresholds 2

Clarifications n EBD errata on book website n SLUBI= Self limited undiagnosed benign illness – not a term I use with parents 3

Definitions: Sensitivity and Specificity 4 Disease status Has diseas e No disease Total Test Resul t PositiveABA + B NegativeCDC + D TotalA + CB + DA + B + C + D Sensitivity = A/ (A+C) Specificity = D/ (B+D) P.I.D. = Positive in Disease N.I.H.= Negative in Health

Definitions: Positive and Negative Predictive value, Accuracy 5 Disease status Has diseas e No disease Total Test Resul t PositiveABA + B NegativeCDC + D TotalA + CB + DA + B + C + D PPV=A/(A+B) NPV=D/(C+D) n Accuracy = (A + D)/(A + B + C + D) n = (A+D)/N n Accuracy demonstration: screening for brain tumors

Definitions: Pretest (prior) and post-test (posterior) probability 6 Disease status Has diseas e No disease Total Test Resul t PositiveABA + B NegativeCDC + D TotalA + CB + DA + B + C + D Pretest probability = ONLY IF SAMPLING IS “CROSS-SECTIONAL”! (A+C)/(A+B+C+D) Posttest probability =A/(A+B) or C/(C+D)

“Cross-sectional” sampling 7 Disease status Has diseas e No disease Total Test Resul t PositiveABA + B NegativeCDC + D TotalA + CB + DA + B + C + D PPV=A/(A+B) NPV=D/(C+D) n Subjects are sampled randomly or consecutively, so that the proportion with disease (pretest probability, prevalence) is clinically meaningful

“Case-control” sampling 8 Disease status Has diseas e No disease Total Test Resul t PositiveABA + B NegativeCDC + D TotalA + CB + DA + B + C + D PPV=A/(A+B) NPV=D/(C+D) n Subjects with and without disease are sampled separately n Proportion with disease is determined by investigator Disease status Has diseas e No disease Total Test Resul t PositiveABA + B NegativeCDC + D TotalA + CB + DA + B + C + D

Prevalence vs Pretest probability n Pretest probability is the more general term n For screening tests, pretest probability = prevalence n For diagnostic tests, pretest probability incorporates history and physical exam items 9

Post-test probability vs. Predictive value n Posttest probability after a + test is the same as positive predictive value n Posttest probability after a – test is 1– negative predictive value 10

2  2 Table Method n Research vignette “Tom, you need to call this mother. She’s really upset.” 11

Choroid Plexus Cyst 12

Fill in table 13 Pretest probability Sensitivity 33% n Specificity 98.5% Disease status Trisomy 18 No Trisomy 18 Total Choroid Plexus Cyst Present Absent Total

Likelihood Ratios 14

Likelihood ratios n A ratio of likelihoods: P(Result|Disease) P(Result|No Disease) n WOWO = With Over WithOut n Pretest odds x LR = Posttest odds n (Prior odds x LR = Posterior odds) 15

What Tests Do Their results change the probability of disease Negative testPositive test Reasurance Treatment Order a Test A good test moves us across action thresholds. 0%100% HIV+HIV- 16

Likelihood of Disease Depends on 2 Things 1. Where you started from (low, medium, high risk) 2. Length and direction of the “arrow” n Basic paradigm: –What we thought before  test result  what we think now 17

Likelihood ratioEffect of test result Very small (0.01)Greatly decreases P(disease) Less than 1 (0.5)Decreases P(disease) OneNo effect on P(disease) More than 1 (2)Increases P(disease) Very big (100)Greatly increases P(disease) 18

Likelihood Ratios n Advantages – Calculation of post-test probability easier (especially when disease is rare) –Capture information for multi-level and continuous tests (next week) n Disadvantages –If either pretest or posttest probability is high (~> 10%) you need to use odds (or a slide rule or calculator) 19

Switch to board n LR for the choroid plexus cyst example –Dichotomous test def of LR n Probability and odds 20

Can Use Slide Rule 21

False-negative confusion n Sensitivity of rapid strep test is 85% n Therefore, false negative rate is 15% n 15% is too high, so always culture to confirm negative rapid strep tests 22

What’s wrong? StrepNo StrepTotal Rapid Test + TPFPTP+FP Rapid Test - FNTNTN+FN TP+FNFP+TN n 2 definitions of “false negative rate” –Def #1: 1-sensitivity = FN/(TP+FN). This one is easier because it’s (assumed to be) constant. –Def #2: 1 - negative predictive value = FN/(FN+TN). This one is harder because it depends on prior probability, but it is the one that should determine clinical decisions. 23

If prior probability of strep = 20% and specificity is 98% n False negative rate (def #2) = 15/407 = 3.7% n NNC (number needed to culture) = 1/.037 = 27 to identify 1 false negative rapid test. (Pre-test probability of 20%) n At some prior probability of strep, culture after negative quick test is not indicated. 24

25 n Sensitivity 85% n Specificity 98% n Prior probability = 20% n Rapid test is NEGATIVE n LR = Try it with slide rule!

Similar examples: n Sensitivity of UA for UTI is only 80%, therefore always culture after a negative UA n Sensitivity of CT scan for subarachnoid hemorrhage is only 90%, therefore always do LP after a negative CT n False positive confusion is similar: 1- specificity vs. 1-positive predictive value 26

Test/Treat Thresholds No test Treat Test 27

“X-Graph” 28

New “X-Graph” 29

Questions? 30