1. February 13, 1997CWU B.Kovalerchuk1 DESIGN OF CONSISTENT SYSTEM FOR RADIOLOGISTS TO SUPPORT BREAST CANCER DIAGNOSIS

2. February 13, 1997CWU B.Kovalerchuk2 Authors Boris Kovalerchuk Department of Computer Science, Central Washington University, Ellensburg, WA 98926-7520 borisk@cwu.edu Evgenii Vityaev Institute of Mathematics, Russian Academy of Science vityaev@mat.nsk.ru James Ruiz Department of Radiology, Woman’s Hospital, Baton Rouge, LA jrmd@womans.com

3. February 13, 1997CWU B.Kovalerchuk3 Overall purpose  to develop a prototype radiological consultation system.  retrieve the second diagnostic opinion (probable diagnosis) for a given case

4. February 13, 1997CWU B.Kovalerchuk4 Agenda BACKGROUND COMPONENTS OF SYSTEM RESULTS METHOD

5. February 13, 1997CWU B.Kovalerchuk5 1. BACKGROUND In the US breast cancer is #1 cancer in women 182,000 cases in 1995 [Wingo, et al., 1995] Screening mammography is the most effective tool against breast cancer However, intra- and inter- observer variability in mammographic interpretation is significant (about 25%)

6. February 13, 1997CWU B.Kovalerchuk6 Computer-Aided Diagnostic (CAD) systems in Breast Cancer neural networks, nearest neighbor methods, discriminant analysis, cluster analysis, linear programming genetic algorithm decision trees ([F. Shtern, 1996], [SCAR, 1996], [TIWDM, 1996] and [CAR, 1996].

7. February 13, 1997CWU B.Kovalerchuk7 Consistency of diagnosis Linear discriminant analysis gives an equation, which separates benign and malignant classes. Example 0.0670x1-0.9653x2+... represents a case. How would one interpret the weighted number of calcifications/cm2 (x1) plus weighted volume (cm3)(x2)? There is no direct medical sense in this formula.

8. February 13, 1997CWU B.Kovalerchuk8 Basis similar cases experience of many radiologists radiologists’ diagnostic rules data base of previous cases Description of a particular caseDescription of a particular case BI-RADS lexicon of American College of Radiology BI-RADS- breast imaging reporting and data system

9. February 13, 1997CWU B.Kovalerchuk9 Advantages System allows a radiologist to get more important information in comparison with known Computer-Aided Diagnostic (CAD) systems. These advances are based on a new computational intelligence technique Logical Data Analysis (LAD)

10. February 13, 1997CWU B.Kovalerchuk10 Implementation A rule-based prototype diagnostic system. The diagnosis is based on combination of the opinions of radiologists and the statistically significant diagnostic rules extracted from the available data base.

11. February 13, 1997CWU B.Kovalerchuk11 2. COMPONENTS OF SYSTEM Component 1. Data/Image Base: Component 2. Diagnostic Rule Base: Component 3. Diagnosis simulator: Component 4. Consultant:

12. February 13, 1997CWU B.Kovalerchuk12 Components of Consultation System data/Image base Rule base simulator of diagnosis “Consultant”

13. February 13, 1997CWU B.Kovalerchuk13 Component 1. Data/Image Base: Supports extracted features of mammograms, Supports patient clinical records, Supports digital images of mammograms Component 2. Diagnostic Rule Base: Support rules extracted from Data Base Support rules obtained by interviewing radiologists Component 3. Diagnosis simulator: Generates diagnosis for a particular case based on the Diagnostic Rule Base.

14. February 13, 1997CWU B.Kovalerchuk14 Component 2. Diagnostic Rule Base: Rules extracted from Data Base Rules obtained by interviewing radiologists radiologist #1 radiologist #2 radiologist #3 If…..Then… If….. Then... IIf…..Then... If….. Then...

15. February 13, 1997CWU B.Kovalerchuk15 “Consultant” Component 4. Consultant: “Consultant” Simulated diagnosis for case x Similar cases {yi} Simulated diagnosis of other radiologists for {yi} Digital reproduction of the mammograms x and {yi} DB rules applicable for case x Statistical significance of rules Comparison user’s (radiologist’s) diagnosis with simulated diagnosis of other radiologists Comparison of user’s diagnosis with DB diagnosis Rationale of applicable diagnostic rules by experienced radiologists from the RB Consultant

16. February 13, 1997CWU B.Kovalerchuk16 3. RESULTS Diagnostic Rules Acquisition. Expert diagnostic rules were extracted from specially organized interviews of a radiologist (J. Ruiz, MD). Method of minimized interview theory of Monotone Boolean Functions hierarchical approach. [Kovalerchuk, et al, 1996 a,b].

17. February 13, 1997CWU B.Kovalerchuk17 Example of extracted rule RULE 1: IF NUMber of calcifications per cm2 (w1) is large AND TOTal number of calcifications (w3) is large AND irregularity in SHAPE of individual calcifications is marked (y1) THEN highly suspicious for malignancy. Mathematical expression w1&w&3y1=>"highly suspicious for malignancy".

18. February 13, 1997CWU B.Kovalerchuk18 Used features calcification features: 1) the number of calcifications/cm2 ; 2) the volume (in cm3) ; 3) total number of calcifications ; 4) irregularity in shape of individual calcifications; 5) variation in shape of calcifications ; 6) variation in size of calcifications; 7) variation in density of calcifications ; 8) density of calcifications; 9) ductal orientation; 10) comparison with previous exam ; 11) associated findings.

19. February 13, 1997CWU B.Kovalerchuk19 Questioning procedure for rule extraction To restore all diagnostic rules thousands of questions might be needed if questions are not specially organized. For 11 diagnostic features of clustered calcifications there are (2 =2,048) feature combinations, representing cases. 11

20. February 13, 1997CWU B.Kovalerchuk20 Results of questioning Only about 40 questions, i.e. 50 times fewer questions than the full set of feature combinations [Kovalerchuk et al, 1996 a,b]. Practically all studies in CAD systems derive diagnostic rules using significantly less than 1,000 cases [Gurney, 1994]. This is the first attempt to work with large number of cases (2,000).

21. February 13, 1997CWU B.Kovalerchuk21 Data used to extract rules 156 cases (77 malignant, 79 benign) 11 features of clustered calcification listed above two extra features: Le Gal type and density of parenchyma the diagnostic classes: "malignant" and "benign".

22. February 13, 1997CWU B.Kovalerchuk22 Diagnostic Rules extracted from Data Base 44 statistically significant diagnostic rules conditional probability greater than 0.7 30 rules with the conditional probability greater than 0.85 18 rules with conditional probability more than 0.95.

23. February 13, 1997CWU B.Kovalerchuk23 Accuracy 44 rules. The total accuracy of diagnosis-- 82%. The False/negative rate -- 6.5% (9 malignant cases were diagnosed as benign) The false/positive rate -- 11.9% (16 benign case were diagnosed as malignant). For the 30 more reliable rules we obtained 90% total accuracy, For the 18 most reliable rules we obtained 96.6% accuracy with only 3 false positive cases (3.4%).

24. February 13, 1997CWU B.Kovalerchuk24 Accuracy Neural Network ("Brainmaker") software 100% accuracy on training data Round-Robin test the total accuracy fell to 66%. The main reason for this low accuracy is that NN do not have a mechanism to evaluate statistical significance /reliability of the performance on training data.

25. February 13, 1997CWU B.Kovalerchuk25 Accuracy Linear Discriminant Analysis ("SIGAMD" software). Poor results (76% on training data test) Decision Tree approach ("SIPINA" software) accuracy of 76%-82% on training data. This is worse than what we obtained for the LAD method with the much more difficult Round-Robin test.

26. February 13, 1997CWU B.Kovalerchuk26 Decision trees x1 < 3 x2>5x2<5 t f restricted different structures no loops Statistical significance? x2 <4 = 8 > 8 8 < x1

27. February 13, 1997CWU B.Kovalerchuk27 Accuracy The extremely important false-negative rate 3-8 cases (LAD), 8-9 cases (Decision Tree), 19 cases (Linear Discriminant Analysis) 26 cases (Neural Network).

28. February 13, 1997CWU B.Kovalerchuk28 Consistency of diagnosis Only LAD and decision trees produce diagnostic rules. These rules make a CAD decision process visible to radiologists. With these methods radiologists can control the decision making process.

29. February 13, 1997CWU B.Kovalerchuk29 CONCLUSION Our study has shown that used Logical Data analysis approach is appropriate for designing a consultation diagnostic system under presented requirements. This approach can be used for development of a full-size consultation system.