1. Lecture 4: Assessing Diagnostic and Screening Tests
Reading:
Gordis - Chapter 4
Lilienfeld and Stolley - Chapter 6, pp

2. Screening
“Screening is the application of a test to people who are asymptomatic for the purpose of classifying a person with respect to their likelihood of having a particular disease”
Screening, in and of itself, does not diagnose disease.
Persons who test positive are referred to physicians for more detailed assessment
Physicians determine the presence or absence of disease.
Screening is one of the most practical applications of epidemiology. It’s goal is to promote health and prevent disease.

3. Screening Decision Tree Negative result Perform screening test Record
the
result
Inform the
person
screened
Positive result
Negative
result
Perform
diagnostic
test
Record
the
result
Inform
the
patient
Positive result
Revise
treatment
and
reevaluate
Start
treatment
Negative
response
Screening
Decision
Tree
Positive response
Continue
treatment
and
reevaluate

4. Outcomes in a screening test
False positive – when a screening test indicates that the individual has a disease but the person in fact does not have the disease.
False negative – when a screening test indicates that the individual does not have a disease but the person in fact has the disease.
True positive – when the test says the person has a disease and the person indeed has the disease.
True negative – when the test says the person does not have the disease and the person in fact is disease free.

5. Screening tests
Validity of test is shown by how well the test actually measures what it is supposed to measure. Validity is determined by the sensitivity and specificity of the test.
Reliability is based on how well the test does in use over time - in its repeatability.

6. Sensitivity and specificity: tests of validity
Sensitivity is the ability of a screening procedure to correctly identify those who have the disease--the percentage of those who have the disease and are proven to have the disease as demonstrated by a diagnostic test.
Specificity is the ability of a screening procedure to correctly identify the percentage of those who do not have the disease--those who do not have the disease and are proven to not have the disease as demonstrated by a diagnostic test.

7. Diagnosed disease status
Screening
Diagnosed disease status
Positive
Negative
Total
Screening test
a=true positive
b=false positive
a+b
c=false negative
d=true negative
c+d
a+c
b+d
Sensitivity = a
b+c
Specificity = d
b+d

8. Sensitivity and specificity of breast cancer screening examination
Cancer confirmed
Cancer not confirmed
Total
Screening test
Positive
132
983
1155
Negative 45
63,650
63,695
177
64,633
64,820
Sensitivity =
132/177 = 74.6%
Specificity =
63650/64633 = 98.5%

9. Screening
Positive predictive value – Probability that a person actually has the disease given a positive screening test
Negative predictive value – Probability that a person is actually disease-free given a negative screening test

10. Screening Positive predictive value = Negative predictive value = a d
Diagnosed disease status
Positive
Negative
Total
Screening test a b
a+b c d
c+d
a+c
b+d
Positive predictive value = a
a+b
Negative predictive value = d
c+d

11. Prevalence on positive predictive value with constant sensitivity and specificity
PV+
Sensitivity
Specificity
(%)
0.1
1.8 90 95
1.0
15.4
5.0
48.6
50.0
94.7
The higher the prevalence, the higher the predictive value.
Screening is most productive if it is applied to a high-risk population.

12. Cutoff level and validity
When the test is a continuous variable, we need a cutoff level to decide positive or negative test result.
If increase the sensitivity by lowering the the cutoff level, we decrease the specificity.

15. Choice of cutoff
The choice of cutoff level depends on the importance attached to false positives and false negatives.
False positives associated with costs – emotional and financial;
false negative associated with missing early detection.

16. How do we examine the reliability (repeatability)?
We do the tests repeatedly in the same individuals and calculate measures of :
Intrasubject variation (variation within individual subjects)
Interobserver variation (variation between those reading the test results)
Overall percent agreement
Kappa statistic

17. Overall percent agreement
Reading No. 2
Abnormal
Suspect
Doubtful
Normal a b c d e f g h i J k l m n o p
Percent agreement =
a+f+k+p
Total reading

18. Kappa statistics We would expect agreement purely by chance.
We want to know:
To what extent do readers agree beyond what we would expect by chance alone?
Answer: calculate Kappa statistics
Kappa =
Observed agreement (%) - agreement expected by chance alone (%)
100% - agreement expected by chance alone (%)

19. Calculate Kappa statistics
Observed table
Expected table
Observer 1 + -
Observer 2 16 2 18 11 27 32 13 45
Observer 1 + -
Observer 2
12.8
5.2 18
19.2
7.8 27 32 13 45
Observed agreement =
(16+11)/45 = 60%
12.8 = 45x(18/45)x(32/45)
7.8 = 45x(27/45)x(13/45)
Expected agreement =
( )/45 = 45.8%
Kappa = (60% %) / (100%-45.8%) = 0.26

20. Interpreting the values of Kappa
Value of Kappa
Strength of agreement
0.0
No agreement
<0.2
Poor
Fair
Moderate
Good
Very good

21. Validity vs reliability
Test results
Reliable but invalid
Valid but not reliable
Both valid and reliable
True value