Step 3: Critically Appraising the Evidence: Statistics for Diagnosis

Table of Contents Clinical Statistics Calculator (Excel) Statistics for: –Diagnosis Sensitivity (Sn) Specificity (Sp) Likelihood Ratios (LR) Pre-test Probabilities/Prevalence Post-Test Probabilities/Predictive Values

Making It Easier If available, find the best evidence in secondary sources where analysis has already occurred. If not pre-assessed, use critical appraisal worksheets to help you through the process.

Importance of Critically Appraising the Evidence Understanding the Limitations of the Author’s Analyses and Interpretations of the Data Assessing Internal Validity Assessing External Validity Identifying Potential Confounding Variables –Simpson’s ParadoxSimpson’s Paradox

Critical Appraisal Basics View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Generalized 2x2 Clinical Table Target Outcome +- Intervention/ Comparison +a (True Positives) b (False Positives) -c (False Negatives) d (True Negatives)

Sensitivity and Specificity View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Diagnostic Table Target DisorderTotal PresentAbsent Diagnostic Test Result +aba + b -cdc + d Totala + cb + da + b + c + d Sensitivity a/(a+c) Specificity d/(b+d)

Sensitivity Answers the question: When the target disorder was present, what fraction of the time did the diagnostic test produce positive results? –Excludes all trials where the diagnostic test produced negative results

Calculating Sensitivity How many times was the target disorder actually present?: a+c How many of these times did the diagnostic test produce positive results?: a –Note: We do not count the number of times the diagnostic test produced positive results when the target disorder was absent (b). We are only interested in the cases where target disorder was present. So what fraction of the total number of times the target disorder was present did the diagnostic test produced positive results?: a / (a + c)

Try it Yourself: Calculating Sensitivity 1 Step 1: How many times was the target disorder actually present? a ca + b a + c

Calculating Sensitivity Bingo! The target disorder was present a+c times.

Calculating Sensitivity Sorry. That is incorrect. The target disorder was present a+c times

Calculating Sensitivity 2 How many of these times did the diagnostic test produce positive results? –Note: We do not count the number of times the diagnostic test produced positive results when the target disorder was absent (b). We are only interested in the cases where target disorder was present.

Try it Yourself: Calculating Sensitivity 2 Step 2: How many of these times did the diagnostic test produce positive results?: a ca + b a + c

Calculating Sensitivity Bingo! The diagnostic test produced positive results a times.

Calculating Sensitivity Sorry. That is incorrect. The diagnostic test produced positive results a times

Try it Yourself: Calculating Sensitivity 3 Step 3: So what fraction of the total number of times the target disorder was present did the diagnostic test produced positive results?: c/(a + c) c/(b + c) a/(a + b) a/(a + c)

Calculating Sensitivity Bingo! The target disorder was present a out of (a + c) times that the diagnostic test produced positive results. So the correct answer is a/(a + c).

Calculating Sensitivity Sorry. That is incorrect. The target disorder was present a out of (a + c) times that the diagnostic test produced positive results. So the correct answer is a/(a + c).

Specificity Answers the question: When the target disorder was not present, what fraction of the time did the diagnostic test produce negative results? –Excludes all trials where the diagnostic test produced positive results

Calculating Specificity 1 How many times was the target disorder absent?: b+d How many of these times did the diagnostic test produce negative results?: d –Note: We do not count the number of times the diagnostic test produced negative results when the target disorder was present (c). We are only interested in the cases where the target disorder was absent. So what fraction of the total number of times the target disorder was absent did the diagnostic test produced negative results?: b/(b+d)

Helpful Acronyms Specificity – SpPin (spin) –When a test has a high Specificity, a Positive test rules in the diagnosis. Sensitivity – SnNout (snout) –When a test has a high Sensitivity, a Negative test rules out the diagnosis.

Another Way of Looking at It Specificity –1 - α –Probability of not getting a Type I Error/ α Error/False Positive Sensitivity –1 - Β –Probability of not getting a Type II Error/ Β Error/False Negative –Also know as the statistical power of the test

Sensitivity/Specificity Example View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Likelihood Ratios Likelihood Ratio for a Positive Result (LR+) –The number of patients testing positive for the target disorder who have the disease for every patient testing positive that does not have it Likelihood Ratio for a Negative Result (LR-) –The number of patients testing negative for the target disorder who have the disease for every patient testing negative that does not have it

Calculating Likelihood Ratios Likelihood Ratio for a Positive Result (LR+) –sensitivity/(1-specificity) Likelihood Ratio for a Negative Result (LR-) –(1-sensitivity)/specificity

Likelihood Ratios Video View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

How to use Likelihood Ratios View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Likelihood Ratio Example View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Prevalence and Probability View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Prevalence Fraction of the population at risk at a given time who have the target disease

Pre-Test Probability Probability that a patient has the target disease without knowing the results of a diagnostic test

Calculating Prevalence (a+c)/(a+b+c+d) The number of those with the disease divided by the total number of people

Post-Test Probabilities Positive Predictive Value –Probability that a patient with a positive diagnostic test actually has the disease Negative Predictive Value –Probability that a patient with a negative diagnostic test actually does not have the disease

Calculating Predictive Values Positive Predictive Values (+PV) – a/(a+b) Negative Predictive Values (-PV) –c/(c+d)

Predictive Values View movie as: –QuickTime (.mov)QuickTime (.mov) –Flash (.swf)Flash (.swf) Double-click on video for full-screen mode.

Try it on your own. Critical Appraisal Practice Exercises –From CEBM

Links to Other Websites and Hands-On Activities EBM Glossary –From CEBM Critical Appraisal Practice Exercises –From CEBM

Congratulations! You have successfully completed Step 3. The End