1. Appraising A Diagnostic Test
Clinical Epidemiology and Evidence-based Medicine Unit
FKUI-RSCM

2. What is diagnosis ?
Increase certainty about presence/absence of disease
Disease severity
Monitor clinical course
Assess prognosis – risk/stage within diagnosis
Plan treatment e.g., location
Stalling for time!
Knottnerus, BMJ 2002

3. Key Concept Pre-test Probability
The probability of the target condition being present before the results of a diagnostic test are available.
Post-test Probability
The probability of the target condition being present after the results of a diagnostic test are available.

4. Key Concept Pre-test Probability Post-test Probability
The probability of the target condition being present before the results of a diagnostic test are available.
Post-test Probability
The probability of the target condition being present after the results of a diagnostic test are available. 4

6. Basic Principles (1) Ideal diagnostic tests – right answers:
(+) results in everyone with the disease and
( - ) results in everyone else
Usual clinical practice:
The test be studied in the same way it would be used in the clinical setting
Observational study, and consists of:
Predictor variable (test result)
Outcome variable (presence / absence of the disease)

7. Basic Principles (2) Sensitivity, specificity
Prevalence, prior probability, predictive values
Likelihood ratios
Dichotomous scale, cutoff points (continuous scale)
Positive (true and false), negative (true & false)
ROC (receiver operator characteristic) curve

8. General structure : 2 X 2 table
Target disorder
Positive
(disease)
Negative
(normal)
Predictor
Test
positive
True positive TP a
False positive FP b
negative
False negative FN c
True negative TN d

9. Disease
(+)
(-)
Total
Test (+)
True pos a
False pos b
a+b
Test (-)
False neg c
True neg d
c+d
a+c
b+d
a+b+c+d
a+c
a+b+c+d
Prevalence
Pretest probability

10. Sensitivity
The proportion of people who truly have a designated disorder who are so identified by the test.
Sensitive tests have few false negatives.
When a test with a high Sensitivity is Negative, it effectively rules out the diagnosis of disease. SnNout

11. Specificity
The proportion of people who are truly free of a designated disorder who are so identified by the test.
Specific tests have few false positives
When a test is highly specific, a positive result can rule in the diagnosis. SpPin

12. Disease (+) (-) Totals a b a+b c d c+d a+c b+d +c+d
Test (+) a b
a+b
Test (-) c d
c+d
a+c
b+d
+c+d
SpPIn
SnNOut
d/b+d
a/a+c
Sensitivity
Specificity
Probability of negative test result
in patients without the disease
Probability of positive test result
in patients with the disease

13. SnNOut
The sensitivity of dyspnea on exertion for the diagnosis of CHF is 100% (41/(41+0)), and the specificity 17% (35/(183+35)).
If DOE, it is very unlikely that they have CHF (0 out of 41 patients with CHF did not have this symptom).
"SnNOut", which is taken from the phrase: "Sensitive test when Negative rules Out disease".

14. SpPin
Conversely, a very specific test, when positive, rules in disease. "SpPIn"! 
The sensitivity of gallop for CHF is only 24% (10/41), but the specificity is 99% (215/218).  Thus, if a patient has a gallop murmur, they probably have CHF (10 out of 13).

15. Iron deficiency anemia result (Serum ferritin)
Sensitivity=a/a+c=90%
Specificity =d/b+d=85%
LR + = sn/(1-sp)=90/15=6
Pos predictive value=a/a+b=73%
Neg predictive value=d/c+d=95%
Prevalence=
(a+c)/(a+b+c+d)= 32%
Iron deficiency anemia
Totals
Present
Absent
Diag
nostic
test
result (Serum ferritin)
(+)
<65 mmol/L
731 a
270 b
1001
a+b
(-)
>65 mmol/L 78 c
1500 d
1578
c+d
809
a+c
1770
b+d
2579
a+b+c+d
Outcome
Predictor
Posttest odd =
Pretest odd x
Likelihood Ratio

16. Odds = ratio of two probabilities
Odds = p/1-p
Probability = odds/1+odds
Likelihood ratio (+):
Prob (+) result in people with the disease
Prob (+) result in people w/out the disease
Pretest Odds X LR = Posttest Odds

17. probability of a test result in pts with disease
Key Concept
Likelihood Ratio
Relative likelihood that a given test would be expected in a patient with (as opposed to one without) a disorder of interest.
probability of a test result in pts with disease
LR=
probability of the test result in pts without disease

18. Likelihood ratios (LR) General Rules of Thumb
LR > 10 or < 0.1 produce large changes in pre-test probability
LR of 5 to 10 or 0.1 to 0.2 produce moderate changes
LR of 1 to 2 or 0.5 to 1 produce small changes in pre-test probability

19. Likelihood ratio Test Test A B C
(1-Sn)/Sp= -
+ = Sn/(1-Sp)
do not
test
get on with treatment
do not
test
treat
Test
Test A B C
pretest probability
posttest probability
PreTest odds x LR
pretest probability

20. The usefulness of five levels of a diagnostic test result
Serum ferritin (mmol/L)
Iron def positive
Iron def
negative
Likelihood ratio
Diagnostic impact No %
Very positive
<15
474 59
(474/809) 20
1.1
(20/1770) 52
Rule in
SpPin
Moderately positive
15-34
175 22
(175/809) 79
4.5
(79/1770)
4.8
Intermed
High
Neutral
35-64 82 10
(82/809)
171
(171/1770) 1
Indeter mine
Moderately negative
65-94 30
3.7
(30/809)
168
9.5
(168/1770)
0.39
Intermed low
Extremely negative
>95 48
5.9
(48/809)
1332 75
(1332/1770)
0.08
Rule out
SnNout
809
100
(809/809)
1770
(1770/1770)

21. Pretest probability
Likelihood ratio
Posttest probability

22. T4 level in suspected hypo- thyroidism in children
T4 value
Hypo
thyroid Eu
5 or less 18 1
5.1 – 7.0 7 17
7.1 – 9.0 4 36
9 or more 3 39
Totals 32 93
T4 value
Hypo
thyroid Eu
≤ 5 18 1
> 5 14 92
Totals 32 93
T4 value
Hypo
thyroid Eu
≤ 7 25 18
> 7 7 75
Totals 32 93
T4 level in suspected hypo-
thyroidism in children
Cutoff point
Sens
Spec 5
0.56
0.99 7
0.78
0.81 9
0.91
0.42
T4 value
Hypo
thyroid Eu
≤ 9 29 54
> 9 3 39
Totals 32 93
For tests / predictors with continuous values result , cutoff points should be determine to choose the best value to use in distinguishing those with and without the target disorder

23. Cutoff point Sens Spec Cutoff point Sens TP 1-Spec FP5
0.56
0.99 7
0.78
0.81 9
0.91
0.42
Cutoff point
Sens TP
1-Spec FP 5
0.56
0.01 7
0.78
0.19 9
0.91
0.58

24. Accuracy of the test
The accuracy of the test depends on how well the test separates the group being tested into those with and without the disease in question
Accuracy is measured by the area under the ROC curve. An area of 1 represents a perfect test; an area of 0.5 represents a worthless test (AUC)
= excellent (A)
= good (B)
= fair (C)
= poor (D)
= fail (F)

25. An ROC curve demonstrates several things:
It shows the tradeoff between sensitivity and specificity
any increase in sensitivity will be accompanied by a decrease in specificity
The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the test.
The closer the curve comes to the 45-degree diagonal of the ROC space, the less accurate the test.
The slope of the tangent line at a cutoff point gives the likelihood ratio (LR) for that value of the test.

26. Appraising DxTest Is the evidence valid? (V)
Was there an independent, blinded comparison with a gold standard?
Was the test evaluated in an appropriate spectrum of patients?
Was the reference standard applied regardless of the test result?
Was the test validated in a second, independent group of patients?

27. Can I trust the accuracy data?
RAMMbo
Recruitment: Was an appropriate spectrum of patients included? (Spectrum Bias)
Maintainence: All patients subjected to a Gold Standard? (Verification Bias)
Measurements: Was there an independent, blind or objective comparison with a Gold standard?
(Observer Bias; Differential Reference Bias)
Guyatt. JAMA, 1993

28. Critical Appraisal Is this valid test important? (I)
Distinguish between patients with and those without the disease
Two by two tables
Sensitivity and Specificity
SnNOut
SpPIn
ROC curves
Likelihood Ratios

30. Critical Appraisal Can I apply this test to my patient (A)
Similarity to our patient
Is it available
Is it affordable
Is it accurate
Is it precise

31. Critical Appraisal Can I apply this test to my patient?
Can I generate a sensible pre-test probability
Personal experience
Practice database
Assume prevalence in the study

32. Critical Appraisal Diagnosis
Can I apply this test to a specific patient
Will the post-test probability affect management
Movement above treatment threshold
Patient willing to undergo testing

33. Thank You