1. CT image testing

2. What is a CT image? CT= computed tomography CT= computed tomography Examines a person in “slices” Examines a person in “slices” Creates density-based images Creates density-based images

3. Three different window densities Bone- densest only bone shows detail, the rest is too dark to make out Bone- densest only bone shows detail, the rest is too dark to make out Soft tissue- moderate density, bone is white, soft tissue is detailed, and “air space” is too dark Soft tissue- moderate density, bone is white, soft tissue is detailed, and “air space” is too dark Lung- low density, bone and soft tissue white, and “air space” is detailed Lung- low density, bone and soft tissue white, and “air space” is detailed

4. Bone Window

5. Soft Tissue Window

6. Lung Window

7. What a radiologist has to look at

8. How diagnoses are decided Look for “abnormalities” in density Look for “abnormalities” in density

9. Why is this cumbersome?

10. Our project- dare to compare Test the new “unified LSU CT image” Test the new “unified LSU CT image”

11. Hypothesis Unified LSU CT image: Unified LSU CT image: Equally accurate Equally accurate Faster Faster

12. Methods Our test Our test ROC Analysis ROC Analysis ROCKit Software ROCKit Software

13. ROC Receiver Operating Characteristics Receiver Operating Characteristics Developed in 1950’s Developed in 1950’s Statistical decision theory Statistical decision theory Used in business, economics, etc Used in business, economics, etc Goal: Is to be a meaningful tool for judging the performance of a diagnostic imaging system

14. Example X  Population X  Population y  healthy z  diseased Gaussian distribution Gaussian distribution Decision boundary Decision boundary Interested in optimum decision boundary Interested in optimum decision boundary

15. EXAMPLE Non-diseased cases Diseased cases Threshold

16. EXAMPLE Non-diseased cases Diseased cases more typically:

17. DECISION BOUNDARY TRUE NEGATIVE TRUE NEGATIVE TRUE POSITIVE TRUE POSITIVE FALSE NEGATIVE FALSE NEGATIVE FALSE POSITIVE FALSE POSITIVE What happens when you place the decision boundary? What happens when you place the decision boundary?

18. DECISION BOUNDARY Threshold Non- diseased cases Diseased cases FALSE NEGATIVE IN RED TRUE POSITVE IN GREEN

19. DECISION BOUNDARY Threshold Non-diseased cases Diseased cases FALSE POSITIVE IN WHITE TRUE NEGATIVE IN WHITE

20. SENSITIVITY TPF Threshold Non-diseased cases Diseased cases SENSITIVITY = TP / TP + FN GREEN /GREEN+RED FNF = 1-TPF

21. SPECIFICITY TNF Threshold Non-diseased cases Diseased cases SPECIFICITY = TN / TN + FP LARGE WHITE / LARGE WHITE + SMALL WHITE FPF = 1-TNF

22. PREVALENCE One of the important properties of ROC is that it is prevalence independent One of the important properties of ROC is that it is prevalence independent Low prevalence is rare Low prevalence is rare Example Example ROC IS NOT PREVALENCE DEPENDENT! ROC IS NOT PREVALENCE DEPENDENT!

23. ROC CURVE TPF, sensitivity FPF, 1-specificity Entire ROC curve

24. COMPARING CURVES

25. ROC EXPERIMENT Datasets Datasets Two options Two options BINARY BINARY MULT-SCALE RATING MULT-SCALE RATING Fitting curves Fitting curves

26. Rockit Who developed Rockit? -Charles E. Metz @University of Chicago What is Rockit? -curve fitting software calculates points for a Roc curve -calculates max probability of two Gaussian distributions What is a Roc Curve? -describes how good the imaging systems is and accuracy of the given data and viewer’s results

27. Example 1 100 patients, 40 negative, 60 positive 100 patients, 40 negative, 60 positiveGiven PPNPNNP… Confidence Rating Confidence Rating 1. Definitely Negative 2. Possibly Negative 3. Not sure 4. Possibly Positive 4. Possibly Positive 5. Definitely Positive This is the viewer’s results 5424123…. Now, from the confidence rating we consider 1=N and 2, 3, 4, 5=P to interpret view’s results. Interpretation of viewer’s result PPPPNPP… Comparing given data and interpretation TPTNFPFN 45 25 0 13 45 25 0 13 (a) TP- disease is present, diagnosed correctly (b) TN-disease is not present, diagnosed correctly (c) FP-person been diagnosed not having the disease, but the disease is present (d) FN-person been diagnosed having the disease, but the disease is not present. Now, we compute the FPF and TPF. In order to compute the FPF, you need this formula FPF=1-TP/TP+FN and TPF=1-TN/TN+FP. Now, we compute the FPF and TPF. In order to compute the FPF, you need this formula FPF=1-TP/TP+FN and TPF=1-TN/TN+FP. Therefore, in our case FPF=1-45/45+13=.224 and TPF=1-25/25+0=0 Therefore, in our case FPF=1-45/45+13=.224 and TPF=1-25/25+0=0 So, our first point is (.224, 0). So, our first point is (.224, 0).

28. Example 2 Now we can compute a second point on the Roc curve. Now we can compute a second point on the Roc curve. We consider a different confidence rating. We consider a different confidence rating. 1, 2=N and 3, 4, 5=P 1, 2=N and 3, 4, 5=P Results based on viewer’s answer from above PPNPNNNN…. The new Truth table is: The new Truth table is: TPTNFPFN 46 28 0 15 46 28 0 15 Now the FPF=1-46 / 46+15 =.245 and TPF= 1-28/ 28+0 = 0 Now the FPF=1-46 / 46+15 =.245 and TPF= 1-28/ 28+0 = 0 Now, we have our second point (.245, 0) Now, we have our second point (.245, 0) Based upon the viewer’s results, a third and forth point can be created by changing the confidence ratings. Based upon the viewer’s results, a third and forth point can be created by changing the confidence ratings. 1, 2, 3=N and 4, 5=P->third point 1, 2, 3, 4=N and 5=P->forth point Therefore, five confidence ratings give you four points on the Roc curve. Therefore, five confidence ratings give you four points on the Roc curve.

29. Rockit input For Condition 1: d For Condition 1: d Enter the Total Number of Actually-Negative Cases (an integer): 40 Beginning with category 1 and separated by blanks, Enter (on one line, integers only) the number of responses to Actually- Negative cases in each category: 1 2 3 4 5 1 2 3 4 5 23 10 2 3 2 23 10 2 3 2 For Condition 1: d For Condition 1: d Enter the Total Number of Actually-Positive Cases (an integer): 60 Beginning with category 1 and separated by blanks, Beginning with category 1 and separated by blanks, Enter (on one line, integers only) the number of responses to Actually- Enter (on one line, integers only) the number of responses to Actually- Positive cases in each category: Positive cases in each category: 1 2 3 4 5 1 2 3 4 5 2 2 3 10 43 2 2 3 10 43

30. Rockit a, b parameters a, b parameters - a represents separation of the Gaussian distributions -b represents width of two Gaussian distributions Open excel template to enter a, b parameters Open excel template to enter a, b parameters Roc curve is produced Roc curve is produced