Receiver Operating Characteristic Curve (ROC) The name "Receiver Operating Characteristic" came from "Signal Detection Theory" developed during World War II for the analysis of radar images. Radar operators had to decide whether a blip on the screen represented an enemy target, a friendly ship, or just noise. Signal detection theory measures the ability of radar receiver operators to make these important distinctions. Their ability to do so was called the Receiver Operating Characteristics.
ROC curves were developed in the 1950's as a by- product of research into making sense of radio signals contaminated by noise. During 1970's that signal detection theory was recognized as useful for interpreting medical test results. More recently it's become clear that they are remarkably useful in medical decision-making
ROC curves plot the sensitivity of a test versus its false positive rate for various points ROC analysis has wide applicability in radiology research for comparing observers, modalities, and tests.
T4 valueHypothyroidEuthyroid 5 or less181 5.1 - 7717 7.1 - 9436 9 or more339 Totals:3293 T4 valueHypothyroidEuthyroid 7 or less2518 > 7775 Totals:3293 T4 valueHypothyroidEuthyroid 5 or less181 > 51492 Totals:3293
The sensitivity and specificity of a diagnostic test depends on more than just the "quality" of the test. They also depend on the definition of what constitutes an abnormal test. No test distinguish normal from disease with 100% accuracy. The area of overlap indicates where the test cannot distinguish normal from disease.
This graph shows three ROC curves representing excellent, good, and worthless tests plotted on the same graph. The accuracy of the test depends on how well the test separates the group being tested into those with and without the disease in question. Accuracy is measured by the area under the ROC curve. An area of 1 represents a perfect test; an area of.5 represents a worthless test. A rough guide for classifying the accuracy of a diagnostic test is the traditional academic point system: 0.90 – 1.00 = excellent 0.80 - 0.90 = good 0.70 - 0.80 = fair 0.60 - 0.70 = poor
Introduction Cardiac troponin testing is central to the diagnosis of acute myocardial infarction An early diagnosis of AMI facilitates rapid decision making and treatment and therefore improves the outcome in patients presenting with chest pain. Introduction of cardiac markers is a milestone in cardiology. The sensitivity of the available cardiac markers is weak within the first hours after the onset of chest pain
???????? Is Sensitive trop sensitive enough?
Methods Study Population. They enrolled 1818 consecutive patients presenting with new-onset chest pain at chest-pain units at three German study centers between January 2007 and December 2008 Study was approved by local ethics committees All patients provided written informed consent
Inclusion: 1.Age between 18 to 85 years 2.Angina pectoris or equivalent Exclusion criteria 1.Major surgery or trauma within the previous 4 weeks 2.Pregnancy 3.IV drug abuse 4.Hemoglobin level <10 g/dL.
Blood was drawn for routine blood work and sample storage at admission, 3 and 6 hours after Concordantly, a 12-lead ECG was obtained. All patients were followed up 30 days after initial hospitalization Additionally, the local civil registry office provided information about death. Follow-up information was available in 98% of the population.
Outcome measures: composite of death. MI. Stroke. Hospital admission because of cardio-vascular reasons. Need for unplanned coronary intervention
Chest pain onset time was carefully assessed by independent research staff. Troponin I Ultra was purchased from Siemens Healthcare Diagnostics, which had no role in the design of the study or analysis of the results.
In-house troponin TropT (Roche Diagnostics, Germany) or Trop I (Siemens Healthcare Diagnostics). Detection limits are 0.01 ng/mL (Troponin T) and 0.04 ng/mL (Troponin I) with assay ranges of 0.01-25 ng/mL (Troponin T) and 0.04-40 ng/mL (Troponin I), respectively. Reference limits based on the 99th percentile for a healthy population are 0.01 ng/mL (Troponin T) and 0.07 ng/mL (Troponin I) with 10% CV cut-offs of 0.03 ng/mL (Troponin T) and 0.14 ng/mL (Troponin I), respectively.
Laboratory Methods Investigational troponin Investigational troponin I was measured with the TnI-Ultra assay on an ADVIA Centaur XP system (Siemens Healthcare Diagnostics, Germany). The assay range is 0.006-50 ng/mL. The reference limit based on the 99th percentile for a healthy population is 0.04 ng/mL
Adjudication of the Final Diagnosis Based on all available clinical, laboratory, and imaging findings Adjudicated by an expert committee of two independent cardiologists who were unaware of the results of the troponin I assays
Diagnosis Based on Conventional Troponin Assays Primary diagnosis of AMI was adjudicated according to current guidelines. Evidence of myocardial necrosis that was consistent with myocardial ischemia, together with clinical symptoms of ischemia or ECG changes indicative of new ischemia (new ST-segment or T-wave changes or new LBBB) or Imaging evidence of new loss of viable myocardium or Detection of a culprit lesion on coronary angiography
Diagnosis Based on Sensitive Troponin I Assay They used the concentration of 0.04 ng per mL as the upper reference limit and established the diagnosis of myocardial infarction if one value of more than 0.04 ng per mL was documented, combined with a rise or fall in the value of 30% or more within 6 hours after admission.
Statistical Analysis Skewed variables were described by median and interquartile range. Normally distributed variables were characterized by their arithmetic mean and standard deviation. They calculated ROC curves on the basis of the continuously measured biomarker levels by taking every measured biomarker level as a cutoff value and then deriving sensitivity and specificity values from the resulting two-by-two tables for each cutoff value. The diagnosis variable used for the two by two tables contrasted non-coronary chest pain against acute myocardial infarction.
P values are based on the Wald z-test statistic All statistical analyses were performed with the use of R software (version 2.8.1) and SAS software (version 9.2).
Low or Moderate Troponin I Levels 585 pts 86 pts 0-006-0.04 Moder. Elevated. (>0.04 ) without rise or fall (<30%)in repeated testing. 6H6H 115 pt > 0.04 30 pts diagnosed as AMI according to trop T 62 had a distinct non- coronary diagnosis
Short-Term Outcome A troponin I level of more than 0.04 ng per milliliter was independently associated with an increased risk of an adverse outcome at 30 days (hazard ratio, 1.96; 95% confidence interval, 1.27 to 3.05; P=0.003).
Assessment of validity Was there an independent, blind comparison with a reference standard (gold standard) of diagnosis? Was the diagnostic test evaluated in an appropriate spectrum of patients? Were all patients analyzed in the groups to which they were randomized (intention to treat analysis)? Was the test, or group of tests, validated in a second, independent group of patients?
Assessment of importance of the results Are likelihood ratios or data necessary for their calculation provided for the diagnostic test? Target DisorderTotals PresentAbsent Diagnostic Test Result Positive aba+b Negative cdc+d Totals a+cb+da+b+c+d
Sensitivity = a/(a+c) Specificity = d/(b+d) Likelihood ratio for a positive test result = LR+ = sens/(1-spec) Likelihood ratio for a negative test result = LR - = (1-sens)/spec Positive Predictive Value = a/(a+b) Negative Predictive Value = d/(c+d) Pre-test probability (prevalence) = (a+c)/(a+b+c+d) Post-test odds=prevalence/(1-prevlance) x LR Post-test probability = post-test odds/(post-test odds +1)
Assessment of Utility: Will the reproducibility of the test result and its interpretation be satisfactory in my setting? Are the results applicable to my patient? Will the results change my management? Will patients be better off as a result of the test?