Download presentation

1
**TEACHING ABOUT DIAGNOSIS**

Tom Sensky Teaching EBM/EBMH St Hugh’s College Oxford September 2005

2
**BY THE END OF THIS SESSION, YOU SHOULD BE ABLE TO ….**

describe and illustrate key measures of diagnostic test performance describe some less commonly quoted measures of diagnostic test performance represent diagnostic test performance in at least four different ways (five if time allows!)

3
**METHOD 1: NATURAL FREQUENCIES GRID**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Assume that the prevalence of the disease is 4%

4
**Assume that of the 4 people with the disease, 3 are picked up by the test**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test

5
**Assume that of the test is positive for a further 7 people who don’t have the disease**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test

6
**The remainder of the sample are negative on the test**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test

7
SENSITIVITY Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test SENSITIVITY is the proportion of people with the disease correctly identified by the test It measures the proportion of false NEGATIVES

8
**SENSITIVITY In this case, sensitivity is ¾ or 75%**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test In this case, sensitivity is ¾ or 75%

9
SPECIFICITY Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test SPECIFICITY is the proportion of people without the disease correctly identified by the test It measures the proportion of false POSITIVES

10
**SPECIFICITY In this case, specificity is (96-7)/96 or 93%**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test In this case, specificity is (96-7)/96 or 93%

11
**If someone is positive on the test, what are the chances that he/she has the disease?**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Probability = 3/10 = 30% This is the POSITIVE PREDICTIVE VALUE (the value of the test in predicting a positive result)

12
**If someone is negative on the test, what are the chances that he/she does not have the disease?**

Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Person without the disease Person with the disease Person who tests positive Person who tests negative True positive on the test False positive on the test True negative on the test False negative on the test Probability = 89/90 = 99% This is the NEGATIVE PREDICTIVE VALUE (the value of the test in predicting a negative result)

13
**SENSITIVITY, SPECIFICITY AND PREDICTIVE VALUES**

For sensitivity and specificity, the reference variable (‘denominator) is the DISEASE For predictive value, the reference variable (‘denominator’) is the TEST

14
**METHOD 2: NATURAL FREQUENCIES TREE**

Population 100

15
**IN EVERY 100 PEOPLE, 4 WILL HAVE THE DISEASE**

Population 100 Disease + 4 Disease - 96 If these 100 people are representative of the population at risk, the assessed rate of those with the disease (4%) represents the PREVALENCE of the disease – it can also be considered the PRE-TEST PROBABILITY of having the disease

16
**OF THE 4 PEOPLE WITH THE DISEASE, THE TEST WILL DETECT 3**

Population 100 Disease + 4 Disease - 96 In other words, the sensitivity is 75% Test + 3 Test - 1

17
**AMONG THE 96 PEOPLE WITHOUT THE DISEASE, 7 WILL TEST POSITIVE**

In other words, the specificity is 93% Population 100 Disease + 4 Disease - 96 Test + 3 Test - 1 Test + 7 Test - 89

18
**AMONG THOSE WHO TEST POSITIVE, 3 IN 10 WILL ACTUALLY HAVE THE DISEASE**

This is also the POST-TEST PROB- ABILITY of having the disease Population 100 Disease + 4 Disease - 96 POSITIVE PREDICTIVE VALUE = 30% Test + 3 Test + 7 Test - 1 Test - 89

19
**AMONG THOSE WHO TEST NEGATIVE, 89 OF 90 WILL NOT HAVE THE DISEASE**

Population 100 Disease + 4 Disease - 96 Test + 3 Test + 7 NEGATIVE PREDICTIVE VALUE = 99% Test - 1 Test - 89

20
**CONVERSELY, IF SOMEONE TESTS NEGATIVE, THE CHANCE OF HAVING THE DISEASE IS ONLY 1 IN 90**

Population 100 Disease + 4 Disease - 96 Test + 3 Test + 7 Test - 1 Test - 89

21
**PREDICTIVE VALUES AND CHANGING PREVALENCE**

Population 1000 Disease + 4 Disease - 996 Prevalence reduced by an order of magnitude from 4% to 0.4%

22
**PREDICTIVE VALUE AND CHANGING PREVALENCE**

Population 1000 Sensitivity and Specificity unchanged Disease + 4 Disease - 996 Test + 3 Test + 70 Test - 1 Test - 926

23
**POSITIVE PREDICTIVE VALUE AT LOW PREVALENCE**

Population 1000 Previously, PPV was 30% Disease + 4 Disease - 996 POSITIVE PREDICTIVE VALUE = 4% Test + 3 Test + 70 Test - 1 Test - 926

24
**NEGATIVE PREDICTIVE VALUE AT LOW PREVALENCE**

Population 1000 Previously, NPV was 99% Disease + 4 Disease - 996 Test + 3 Test + 70 NEGATIVE PREDICTIVE VALUE >99% Test - 1 Test - 926

25
**PREDICTION OF LOW PREVALENCE EVENTS**

Even highly specific tests, when applied to low prevalence events, yield a high number of false positive results Because of this, under such circumstances, the Positive Predictive Value of a test is low However, this has much less influence on the Negative Predictive Value

26
**RELATIONSHIP BETWEEN PREVALENCE AND PREDICTIVE VALUE**

Difference between PPV and NPV relatively small Difference between PPV and NPV relatively large Based on a test with 90% sensitivity and 82% specificity

27
**RELATIONSHIP BETWEEN PREVALENCE AND PREDICTIVE VALUE**

Based on a test with 75% sensitivity and 93% specificity

28
**PERFORMANCE OF A TEST WITH CHANGING PREVALENCE**

A : Sensitivity = Specificity = 0.9 LR+ = 9.0 B : Sensitivity = Specificity = 0.7 LR+ = 3.0 C : Sensitivity = Specificity = 0.5 LR+ = 1.0 POST-TEST PROBABILITY

29
**LIKELIHOOD Population 100 Disease + 4 Test + 3 Test - 1**

The likelihood that someone with the disease will have a positive test is ¾ or 75% This is the same as the sensitivity Test + 3 Test - 1

30
**LIKELIHOOD II Population 100 Disease - 96 Test + 7 Test - 89**

The likelihood that someone without the disease will have a positive test is 7/96 or 7% This is the same as the (1-specificity) Test + 7 Test - 89

31
**LIKELIHOOD RATIO LIKELIHOOD OF POSITIVE TEST GIVEN THE DISEASE**

= LIKELIHOOD OF POSITIVE TEST IN THE ABSENCE OF THE DISEASE SENSITIVITY 1- SPECIFICITY 0.75 0.07 = = = 10.7 A Likelihood Ratio of 1.0 indicates an uninformative test (occurs when sensitivity and specificity are both 50%) The higher the Likelihood Ratio, the better the test (other factors being equal)

32
**METHOD 3: ‘TRADITIONAL’ 2x2 TABLES**

33
**SENSITIVITY SENSITIVITY**

FALSE NEGATIVES SENSITIVITY The proportion of people with the diagnosis (N=4) who are correctly identified (N=3) Sensitivity = a/(a+c) = 3/4 = 75%

34
**SPECIFICITY SPECIFICITY**

FALSE POSITIVES SPECIFICITY The proportion of people without the diagnosis (N=96) who are correctly identified (N=89) Specificity = d/(b+d) = 89/96 = 93%

35
PRE-TEST ODDS In the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)

36
POST-TEST ODDS In the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS) In those who score positive on the test, the odds of having the disease are 3 to 7 or 43% (the POST-TEST ODDS)

37
POST-TEST ODDS In the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS) In those who score positive on the test, the odds of having the disease are 3 to 7 or 43% (the POST-TEST ODDS) In those who score negative on the test, the odds of having the disease are 1 to 89 or approximately 1%

38
DIAGNOSTIC ODDS RATIO Potentially useful as an overall summary measure, but only in conjunction with other measures (LR, sensitivity, specificity) The Diagnostic Odds Ratio is the ratio of odds of having the diagnosis given a positive test to those of having the diagnosis given a negative test

39
**LIKELIHOOD RATIO x PRE-TEST ODDS**

BAYES THEOREM POST-TEST ODDS = LIKELIHOOD RATIO x PRE-TEST ODDS

40
**LIKELIHOOD RATIO AND PRE- AND POST-TEST PROBABILITIES**

For a given test with a given likelihood ratio, the post-test probability will depend on the pre-test probability (that is, the prevalence of the condition in the sample being assessed)

41
**SENSITIVITY ANALYSIS OF A DIAGNOSTIC TEST**

Value 95% CI Pre-test probability 35% 26% to 44%

42
**SENSITIVITY ANALYSIS OF A DIAGNOSTIC TEST**

Value 95% CI Pre-test probability 35% 26% to 44% Likelihood ratio 5.0 3.0 to 8.5 Applying the 95% confidence intervals above to the nomogram, the post-test probability is likely to lie in the range 55-85%

43
**RECEIVER OPERATING CHARACTERISTIC CURVE**

The diagonal line (representing Sensitivity=0.5 and Specificity=0.5) represents performance no better than chance RECEIVER OPERATING CHARACTERISTIC CURVE Overall shape is predicted by the reciprocal relationship between sensitivity and specificity The closer the curve gets to Sensitivity=1 and Specificity=1, the better the overall performance of the test TRUE POSITIVE RATE (Sensitivity) Hence the area under the curve gives a measure of the test’s performance FALSE POSITIVE RATE (1-Specificity)

44
AREA UNDER ROC CURVES Sensitivity and specificity both 100% - TEST PERFECT AREA=1.0 Sensitivity and specificity both 50% - TEST USELESS The area under a ROC curve will be between 0.5 and 1.0 AREA=0.5

45
AREA UNDER ROC CURVES Area = 0.7 (between 0.5 and 1.0) Consider (hypothetically) two patients drawn randomly from the DISEASE+ and DISEASE- groups respectively If the test is used to guess which patient is from the DISEASE+ group, it will be right 70% of the time

46
**APPLYING A DIAGNOSTIC TEST IN DIFFERENT SETTINGS**

The Positive Predictive Value of a test will vary (according to the prevalence of the condition in the chosen setting) Sensitivity and Specificity are usually considered properties of the test rather than the setting, and are therefore usually considered to remain constant However, sensitivity and specificity are likely to be influenced by complexity of differential diagnoses and a multitude of other factors (cf spectrum bias)

47
**RECEIVER OPERATING CHARACTERISTIC (ROC) CURVE**

This study compared the performance of a dementia screening test in a community sample (ACAT) and a memory clinic sample (MC) Flicker L, Loguidice D, Carlin JB, Ames D. The predictive value of dementia screening instruments in clinical populations. International Journal of Geriatric Psychiatry 1997 ; 12 :

48
**METHOD 4: A TEST WITH NORMALLY DISTRIBUTED VALUES**

Assessing the performance of the test assumes that these two distributions remain constant. However, each of them will vary (particularly through spectrum or selection bias) Test cut-off % of Group NON-DESEASED DISEASED Negative Positive Degree of ‘positivity’ on test

49
**PERFORMANCE OF A DIAGNOSTIC TEST**

NON-CASES CASES FALSE NEGATIVES Test cut-off FALSE POSITIVES % of Group NON-DESEASED DISEASED Negative Positive Degree of ‘positivity’ on test

50
**MINIMISING FALSE NEGATIVES: A SENSITIVE TEST**

NON-CASES CASES Cut-off shifted to minimise false negatives ie to optimise sensitivity CONSEQUENCES: Specificity reduced A Negative result from a seNsitive test rules out the diagnosis - snNout Test cut-off % of Group NON-DESEASED DISEASED Negative Positive Degree of ‘positivity’ on test

51
**MINIMISING FALSE POSITIVES: A SPECIFIC TEST**

Cut-off shifted to minimise false positives ie to optimise specificity CONSEQUENCES: Sensitivity reduced A Positive result from a sPecific test rules in the diagnosis - spPin Test cut-off % of Group NON-DESEASED DISEASED Negative Positive Degree of ‘positivity’ on test

52
**METHOD 5: USING SCALES WITH DIFFERENT CUT-OFFS**

CASES NON-CASES Sensitivity = A/A+C Specificity = D/B+D FALSE POSITIVES POSITIVES TRUE A B MMSE Score C D Chosen cut-off TRUE NEGATIVES NEGATIVES FALSE

53
**INCREASING SENSITIVITY**

CASES NON-CASES Sensitivity = A/A+C Specificity = D/B+D In a seNsitive test, false Negatives are minimised A negative result from a sensitive test rules out the diagnosis (snNnout) TRUE POSITIVES MMSE Score A B C D FALSE NEGATIVES

54
**INCREASING SPECIFICITY**

CASES NON-CASES Sensitivity = A/A+C Specificity = D/B+D FALSE POSITIVES A B In a sPecific test, false Positives are minimised A positive result from a specific test rules in the diagnosis (spPin) C D MMSE Score TRUE NEGATIVES

55
KEY REFERENCES Sedlmeier P and Gigerenzer G. Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General. 130 (3): , Knotternus JA (ed). The Evidence Base of Clinical Diagnosis. London: BMJ Books, 2002. Sackett DL, Haynes RB, Guyatt G, and Tugwell P. Clinical Epidemiology : A Basic Science for Clinical Medicine. Boston, Mass: Little, Brown & Co, 1991. Loong TW. Understanding sensitivity and specificity with the right side of the brain. BMJ 2003: 327:

Similar presentations

Presentation is loading. Please wait....

OK

Statistical Fridays J C Horrow, MD, MSSTAT

Statistical Fridays J C Horrow, MD, MSSTAT

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on next generation 2-stroke engine oil Ppt on end of life care Ppt on history of australia for kids Ppt on object oriented programming with c++ pdf Ppt on fire extinguisher types wiki Presenting ppt on ipad Ppt on cross docking meaning Presentations ppt online maker Ppt on e learning system Ppt on first conditional game