Appraising A Diagnostic Test

Slides:



Advertisements
Similar presentations
Step 3: Critically Appraising the Evidence: Statistics for Diagnosis.
Advertisements

TESTING A TEST Ian McDowell Department of Epidemiology & Community Medicine November, 2004.
Diagnostic Tests Patrick S. Romano, MD, MPH Professor of Medicine and Pediatrics Patrick S. Romano, MD, MPH Professor of Medicine and Pediatrics.
Critically Evaluating the Evidence: diagnosis, prognosis, and screening Elizabeth Crabtree, MPH, PhD (c) Director of Evidence-Based Practice, Quality Management.
Receiver Operating Characteristic (ROC) Curves
Evidence-Based Diagnosis in Physical Therapy Julie M. Fritz, PhD, PT, ATC Department of Physical Therapy University of Pittsburgh.
Rapid Critical Appraisal of diagnostic accuracy studies Professor Paul Glasziou Centre for Evidence Based Medicine University of Oxford
CRITICAL APPRAISAL Dr. Cristina Ana Stoian Resident Journal Club
GerstmanChapter 41 Epidemiology Kept Simple Chapter 4 Screening for Disease.
Vanderbilt Sports Medicine Chapter 4: Prognosis Presented by: Laurie Huston and Kurt Spindler Evidence-Based Medicine How to Practice and Teach EBM.
DIAGNOSTIC TESTS Assist. Prof. E. Çiğdem Kaspar Yeditepe University Faculty of Medicine Department of Biostatistics and Medical Informatics.
EBM-Diagnostic Testing K. Mae Hla, MD, MHS Primary Care Faculty Development Fellowship November 13, 2010.
Interpreting Diagnostic Tests
Diagnosis Concepts and Glossary. Cross-sectional study The observation of a defined population at a single point in time or time interval. Exposure and.
HOW TO READ AN ARTICLE ABOUT A DIAGNOSTIC TEST Chitkara MB, Boykan R, Messina C Stony Brook Long Island Children’s Hospital.
Statistics in Screening/Diagnosis
Medical decision making. 2 Predictive values 57-years old, Weight loss, Numbness, Mild fewer What is the probability of low back cancer? Base on demographic.
Diagnosis Articles Much Thanks to: Rob Hayward & Tanya Voth, CCHE.
DEB BYNUM, MD AUGUST 2010 Evidence Based Medicine: Review of the basics.
Diagnostic Testing Ethan Cowan, MD, MS Department of Emergency Medicine Jacobi Medical Center Department of Epidemiology and Population Health Albert Einstein.
EBMRC How to Use an Article About a Diagnostic Test Akbar Soltani. MD, Endocrinologist Tehran University of Medical Sciences (TUMS) Endocrine and Metabolism.
Division of Population Health Sciences Royal College of Surgeons in Ireland Coláiste Ríoga na Máinleá in Éirinn Indices of Performances of CPRs Nicola.
Vanderbilt Sports Medicine How to practice and teach EBM Chapter 3 May 3, 2006.
Sensitivity Sensitivity answers the following question: If a person has a disease, how often will the test be positive (true positive rate)? i.e.: if the.
1 Interpreting Diagnostic Tests Ian McDowell Department of Epidemiology & Community Medicine January 2012 Note to readers: you may find the additional.
Evidence Based Medicine Workshop Diagnosis March 18, 2010.
Evidence Based Practice in Psychology – Lecture 3 May 31, 2007.
Screening and Diagnostic Testing Sue Lindsay, Ph.D., MSW, MPH Division of Epidemiology and Biostatistics Institute for Public Health San Diego State University.
EBCP. Random vs Systemic error Random error: errors in measurement that lead to measured values being inconsistent when repeated measures are taken. Ie:
EVIDENCE ABOUT DIAGNOSTIC TESTS Min H. Huang, PT, PhD, NCS.
Advanced Physical therapy Procedures Ahmed alhowimel.
Diagnosis: EBM Approach Michael Brown MD Grand Rapids MERC/ Michigan State University.
MEASURES OF TEST ACCURACY AND ASSOCIATIONS DR ODIFE, U.B SR, EDM DIVISION.
Evaluating Diagnostic Tests Payam Kabiri, MD. PhD. Clinical Epidemiologist Tehran University of Medical Sciences.
Likelihood 2005/5/22. Likelihood  probability I am likelihood I am probability.
Evidence-Based Medicine Diagnosis Component 2 / Unit 5 1 Health IT Workforce Curriculum Version 1.0 /Fall 2010.
1 Risk Assessment Tests Marina Kondratovich, Ph.D. OIVD/CDRH/FDA March 9, 2011 Molecular and Clinical Genetics Panel for Direct-to-Consumer (DTC) Genetic.
Stats Facts Mark Halloran. Diagnostic Stats Disease present Disease absent TOTALS Test positive aba+b Test negative cdc+d TOTALSa+cb+da+b+c+d.
DIAGNOSTIC & SCREENING Evidence-based Medicine. Pengalaman/Data Empiric Masalah experience-based medicine Nilai-nilai kebenaran Nilai-nilai pembenaran.
TESTING A TEST Ian McDowell Department of Epidemiology & Community Medicine January 2008.
Prediction statistics Prediction generally True and false, positives and negatives Quality of a prediction Usefulness of a prediction Prediction goes Bayesian.
HSS4303B – Intro to Epidemiology Feb 8, Agreement.
1 Wrap up SCREENING TESTS. 2 Screening test The basic tool of a screening program easy to use, rapid and inexpensive. 1.2.
Diagnostic Tests Studies 87/3/2 “How to read a paper” workshop Kamran Yazdani, MD MPH.
Diagnostic Test Characteristics: What does this result mean
Evidence based medicine Diagnostic tests Ross Lawrenson.
1 Medical Epidemiology Interpreting Medical Tests and Other Evidence.
EVALUATING u After retrieving the literature, you have to evaluate or critically appraise the evidence for its validity and applicability to your patient.
Laboratory Medicine: Basic QC Concepts M. Desmond Burke, MD.
ROC curve estimation. Index Introduction to ROC ROC curve Area under ROC curve Visualization using ROC curve.
Timothy Wiemken, PhD MPH Assistant Professor Division of Infectious Diseases Diagnostic Tests.
PTP 560 Research Methods Week 12 Thomas Ruediger, PT.
Diagnostic Likelihood Ratio Presented by Juan Wang.
Diagnosis:Testing the Test Verma Walker Kathy Davies.
Sensitivity, Specificity, and Receiver- Operator Characteristic Curves 10/10/2013.
Sensitivity:DISEASE Specificity:PRESENTABSENT TEST RESULTS POSITIVEAbA+b NEGATIVECdC+d a+cb+da+b+c+d Lets recall the 2x2 contingency table Probability.
Critical Appraisal Course for Emergency Medicine Trainees Module 5 Evaluation of a Diagnostic Test.
Diagnostic studies Adrian Boyle.
DR.FATIMA ALKHALEDY M.B.Ch.B;F.I.C.M.S/C.M
Diagnostic Test Studies
Sensitivity and Specificity
Evidence-Based Medicine
Principles of Epidemiology E
بسم الله الرحمن الرحيم Clinical Epidemiology
Diagnosis II Dr. Brent E. Faught, Ph.D. Assistant Professor
The receiver operating characteristic (ROC) curve
Refining Probability Test Informations Vahid Ashoorion MD. ,MSc,
کارگاه تکميلی کشوری تربيت مربی آموزش طب مبتنی بر شواهد
Evidence Based Diagnosis
Presentation transcript:

Appraising A Diagnostic Test Clinical Epidemiology and Evidence-based Medicine Unit FKUI-RSCM

What is diagnosis ? Increase certainty about presence/absence of disease Disease severity Monitor clinical course Assess prognosis – risk/stage within diagnosis Plan treatment e.g., location Stalling for time! Knottnerus, BMJ 2002

Key Concept Pre-test Probability The probability of the target condition being present before the results of a diagnostic test are available. Post-test Probability The probability of the target condition being present after the results of a diagnostic test are available.

Key Concept Pre-test Probability Post-test Probability The probability of the target condition being present before the results of a diagnostic test are available. Post-test Probability The probability of the target condition being present after the results of a diagnostic test are available. 4

Basic Principles (1) Ideal diagnostic tests – right answers: (+) results in everyone with the disease and ( - ) results in everyone else Usual clinical practice: The test be studied in the same way it would be used in the clinical setting Observational study, and consists of: Predictor variable (test result) Outcome variable (presence / absence of the disease)

Basic Principles (2) Sensitivity, specificity Prevalence, prior probability, predictive values Likelihood ratios Dichotomous scale, cutoff points (continuous scale) Positive (true and false), negative (true & false) ROC (receiver operator characteristic) curve

General structure : 2 X 2 table Target disorder Positive (disease) Negative (normal) Predictor Test positive True positive TP a False positive FP b negative False negative FN c True negative TN d

Disease (+) (-) Total Test (+) True pos a False pos b a+b Test (-) False neg c True neg d c+d a+c b+d a+b+c+d a+c a+b+c+d Prevalence Pretest probability

Sensitivity The proportion of people who truly have a designated disorder who are so identified by the test. Sensitive tests have few false negatives. When a test with a high Sensitivity is Negative, it effectively rules out the diagnosis of disease. SnNout

Specificity The proportion of people who are truly free of a designated disorder who are so identified by the test. Specific tests have few false positives When a test is highly specific, a positive result can rule in the diagnosis. SpPin

Disease (+) (-) Totals a b a+b c d c+d a+c b+d +c+d Test (+) a b a+b Test (-) c d c+d a+c b+d +c+d SpPIn SnNOut d/b+d a/a+c Sensitivity Specificity Probability of negative test result in patients without the disease Probability of positive test result in patients with the disease

SnNOut The sensitivity of dyspnea on exertion for the diagnosis of CHF is 100% (41/(41+0)), and the specificity 17% (35/(183+35)). If DOE, it is very unlikely that they have CHF (0 out of 41 patients with CHF did not have this symptom). "SnNOut", which is taken from the phrase: "Sensitive test when Negative rules Out disease".

SpPin Conversely, a very specific test, when positive, rules in disease. "SpPIn"!  The sensitivity of gallop for CHF is only 24% (10/41), but the specificity is 99% (215/218).  Thus, if a patient has a gallop murmur, they probably have CHF (10 out of 13).

Iron deficiency anemia result (Serum ferritin) Sensitivity=a/a+c=90% Specificity =d/b+d=85% LR + = sn/(1-sp)=90/15=6 Pos predictive value=a/a+b=73% Neg predictive value=d/c+d=95% Prevalence= (a+c)/(a+b+c+d)= 32% Iron deficiency anemia Totals Present Absent Diag nostic test result (Serum ferritin) (+) <65 mmol/L 731 a 270 b 1001 a+b (-) >65 mmol/L 78 c 1500 d 1578 c+d 809 a+c 1770 b+d 2579 a+b+c+d Outcome Predictor Posttest odd = Pretest odd x Likelihood Ratio

Odds = ratio of two probabilities Odds = p/1-p Probability = odds/1+odds Likelihood ratio (+): Prob (+) result in people with the disease Prob (+) result in people w/out the disease Pretest Odds X LR = Posttest Odds

probability of a test result in pts with disease Key Concept Likelihood Ratio Relative likelihood that a given test would be expected in a patient with (as opposed to one without) a disorder of interest. probability of a test result in pts with disease LR= probability of the test result in pts without disease

Likelihood ratios (LR) General Rules of Thumb LR > 10 or < 0.1 produce large changes in pre-test probability LR of 5 to 10 or 0.1 to 0.2 produce moderate changes LR of 1 to 2 or 0.5 to 1 produce small changes in pre-test probability

Likelihood ratio Test Test A B C (1-Sn)/Sp= - + = Sn/(1-Sp) do not test get on with treatment do not test treat Test Test A B C 0 .10 .20 .30 .40 .50 .60 .70 .80 .90 1 pretest probability posttest probability PreTest odds x LR pretest probability

The usefulness of five levels of a diagnostic test result Serum ferritin (mmol/L) Iron def positive Iron def negative Likelihood ratio Diagnostic impact No % Very positive <15 474 59 (474/809) 20 1.1 (20/1770) 52 Rule in SpPin Moderately positive 15-34 175 22 (175/809) 79 4.5 (79/1770) 4.8 Intermed High Neutral 35-64 82 10 (82/809) 171 (171/1770) 1 Indeter mine Moderately negative 65-94 30 3.7 (30/809) 168 9.5 (168/1770) 0.39 Intermed low Extremely negative >95 48 5.9 (48/809) 1332 75 (1332/1770) 0.08 Rule out SnNout 809 100 (809/809) 1770 (1770/1770)

Pretest probability Likelihood ratio Posttest probability

T4 level in suspected hypo- thyroidism in children T4 value Hypo thyroid Eu 5 or less 18 1 5.1 – 7.0 7 17 7.1 – 9.0 4 36 9 or more 3 39 Totals 32 93 T4 value Hypo thyroid Eu ≤ 5 18 1 > 5 14 92 Totals 32 93 T4 value Hypo thyroid Eu ≤ 7 25 18 > 7 7 75 Totals 32 93 T4 level in suspected hypo- thyroidism in children Cutoff point Sens Spec 5 0.56 0.99 7 0.78 0.81 9 0.91 0.42 T4 value Hypo thyroid Eu ≤ 9 29 54 > 9 3 39 Totals 32 93 For tests / predictors with continuous values result , cutoff points should be determine to choose the best value to use in distinguishing those with and without the target disorder

Cutoff point Sens Spec Cutoff point Sens TP 1-Spec FP 5 0.56 0.99 7 0.78 0.81 9 0.91 0.42 Cutoff point Sens TP 1-Spec FP 5 0.56 0.01 7 0.78 0.19 9 0.91 0.58

Accuracy of the test The accuracy of the test depends on how well the test separates the group being tested into those with and without the disease in question Accuracy is measured by the area under the ROC curve. An area of 1 represents a perfect test; an area of 0.5 represents a worthless test (AUC) 0.90-1.00 = excellent (A) 0.80-0.90 = good (B) 0.70-0.80 = fair (C) 0.60-0.70 = poor (D) 0.50-0.60 = fail (F)

An ROC curve demonstrates several things: It shows the tradeoff between sensitivity and specificity any increase in sensitivity will be accompanied by a decrease in specificity The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the test. The closer the curve comes to the 45-degree diagonal of the ROC space, the less accurate the test. The slope of the tangent line at a cutoff point gives the likelihood ratio (LR) for that value of the test.

Appraising DxTest Is the evidence valid? (V) Was there an independent, blinded comparison with a gold standard? Was the test evaluated in an appropriate spectrum of patients? Was the reference standard applied regardless of the test result? Was the test validated in a second, independent group of patients?

Can I trust the accuracy data? RAMMbo Recruitment: Was an appropriate spectrum of patients included? (Spectrum Bias) Maintainence: All patients subjected to a Gold Standard? (Verification Bias) Measurements: Was there an independent, blind or objective comparison with a Gold standard? (Observer Bias; Differential Reference Bias) Guyatt. JAMA, 1993

Critical Appraisal Is this valid test important? (I) Distinguish between patients with and those without the disease Two by two tables Sensitivity and Specificity SnNOut SpPIn ROC curves Likelihood Ratios

Critical Appraisal Can I apply this test to my patient (A) Similarity to our patient Is it available Is it affordable Is it accurate Is it precise

Critical Appraisal Can I apply this test to my patient? Can I generate a sensible pre-test probability Personal experience Practice database Assume prevalence in the study

Critical Appraisal Diagnosis Can I apply this test to a specific patient Will the post-test probability affect management Movement above treatment threshold Patient willing to undergo testing

Thank You