The False Positive Paradox Do you react when you hear a car alarm? Why not? Approximately 250,000,000 motor vehicles are registered in the U.S. Approximately 700,000 cars are stolen each year, which is 0.3%.

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL COLUMN TOTAL 3 997 1000

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive) Car Alarm Does NOT Sound (Test is Negative) COLUMN TOTAL 3 997 1000

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive) Car Alarm Does NOT Sound (Test is Negative) COLUMN TOTAL 3 997 1000 Sensitivity refers to the True Positives, the proportion of cars being stolen that the car alarm detects accurately. Specificity refers to the True Negatives, the proportion of cars NOT being stolen whose alarms don’t sound.

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive) 3 ~= 99% of 3 “True Positive” Car Alarm Does NOT Sound (Test is Negative) 987 ~= 99% of 997 “True Negative”  COLUMN TOTAL 3 997 1000 Sensitivity refers to the True Positives, the proportion of cars being stolen that the car alarm detects accurately. Specificity refers to the True Negatives, the proportion of cars NOT being stolen whose alarms don’t sound. For our example, let’s make the Sensitivity and Specificity both 99%.

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive)  3 “True Positive” 10 ~= 1% of 997 “False Positive”   13 Car Alarm Does NOT Sound (Test is Negative)  0 ~= 1% of 3 “False Negative”  987 “True Negative” COLUMN TOTAL 3 997 1000 Sensitivity refers to the True Positives, the proportion of cars being stolen that the car alarm detects accurately. Specificity refers to the True Negatives, the proportion of cars NOT being stolen whose alarms don’t sound. For our example, let’s make the Sensitivity and Specificity both 99%. A False Positive occurs when a car alarm sounds but the car is not being stolen. A False Negative occurs when a car alarm does not sound, but the car is being stolen.

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive)  3 “True Positive” 10 “False Positive”   13 Car Alarm Does NOT Sound (Test is Negative)  0 “False Negative”  987 “True Negative” COLUMN TOTAL 3 997 1000 Sensitivity refers to the True Positives, the proportion of cars being stolen that the car alarm detects accurately. Specificity refers to the True Negatives, the proportion of cars NOT being stolen whose alarms don’t sound. For our example, let’s make the Sensitivity and Specificity both 99%. A False Positive occurs when a car alarm sounds but the car is not being stolen. A False Negative occurs when a car alarm does not sound, but the car is being stolen.

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive)  3 “True Positive” 10 “False Positive”   13 Car Alarm Does NOT Sound (Test is Negative)  0 “False Negative”  987 “True Negative” COLUMN TOTAL 3 997 1000 77% (10 of 13) of the car alarms are incorrect! “given” Conditional Probability: Pr(Car Not Stolen | Car Alarm Sounds) = 10/13 This is why medical screenings typically test a “B” sample with a more thorough test. And it is worse for things that rarely occur.

The False Positive Paradox   Car Stolen Car NOT Stolen ROW TOTAL Car Alarm Sounds (Test is Positive)  1 “True Positive” 10 “False Positive”   11 Car Alarm Does NOT Sound (Test is Negative)  0 “False Negative”  989 “True Negative” COLUMN TOTAL 1 999 1000 And it is worse for things that rarely occur. Consider what happens if it is a 0.01% chance (1/3 of our previous percentage) of getting stolen. Now it’s 91% (10 of 11) of the car alarms are incorrect! “given” Conditional Probability: Pr(Car Not Stolen | Car Alarm Sounds) = 10/11