1 ASU MAT 591: Opportunities in Industry! ASU MAT 591: Opportunities In Industry!

2 ASU MAT 591: Opportunities in Industry! Special Processing Algorithms – Part 1 Automatic Target Recognition Special Processing Algorithms – Part 1 Automatic Target Recognition November 4, 2003 Chung-Fu Chang, Ph.D. Lockheed Martin Management and Data Systems ISR Systems 1300 S. Litchfield Road Goodyear Arizona

3 ASU MAT 591: Opportunities in Industry! Agenda Introduction Bayesian neural network Applications –Fingerprint classification –Medical imagery orientation Discussions

4 ASU MAT 591: Opportunities in Industry! Introduction Why do we need ATR? –Too much information, not enough personnel Who are the users? –Image analysts, fingerprint experts, radiologists, etc. What is the underlining technologies? –Rule-based (AI, Expert Systems, etc.) –Computer vision (Pattern matching) –Statistical approach (Bayesian classifier) –Neural Network  Feedback learning How to train the machine/algorithm? –Supervised learning –Unsupervised learning

5 ASU MAT 591: Opportunities in Industry! is a posteriori probability is a priori probability The key is to closely estimate the probability density function,. Bayes Rule

6 ASU MAT 591: Opportunities in Industry! Example of Two Object Types Feature Measurement Probability Density Function Object 1 Object 2 x

7 ASU MAT 591: Opportunities in Industry! Automatic Target Recognition Synthetic Aperture Radar (SAR) Signatures with statistical variation –Target representation –Target environment Optimal approach –Bayesian classification –Neural Network paradigm

8 ASU MAT 591: Opportunities in Industry! Overview of ATR Algorithm

9 ASU MAT 591: Opportunities in Industry! Overview of ATR process

10 ASU MAT 591: Opportunities in Industry! SAR Feature Investigated Mean Standard Deviation Spatial information Edge Magnitude Peak Side lobe Impulse response Polarization Height

11 ASU MAT 591: Opportunities in Industry! Neural Network Paradigm

12 ASU MAT 591: Opportunities in Industry! Feature Distribution Estimation f1f1 f2f2 f1f1 f2f2

13 ASU MAT 591: Opportunities in Industry! BNN Equations

14 ASU MAT 591: Opportunities in Industry! BNN Equations - continued

15 ASU MAT 591: Opportunities in Industry! BNN Equations - continued

16 ASU MAT 591: Opportunities in Industry! BNN Equations - continued

17 ASU MAT 591: Opportunities in Industry! BNN Paradigm... Feature Components Likelihood Functions Cluster NodesObject Type Nodes K K p ( f | O ) P( O )... One Cluster All the cluster nodes representing clusters which belong to an object type will connect to the node representing that object type

18 ASU MAT 591: Opportunities in Industry! BNN Implementation... Feature ComponentsLikelihood Functions Cluster NodesObject Type Nodes k nn L ( f | O )... One Cluster F 2 F 3 F 1 b j k c i jk m i,  i All the cluster nodes representing clusters which belong to an object type will connect to the node representing that object type

19 ASU MAT 591: Opportunities in Industry! Training and Learning

20 ASU MAT 591: Opportunities in Industry! a posteriori Probability & Decision fBNN P r ( fO1O1 ) fO2O2 ) fO3O3 ) Feature Vector a posteriori probabilities O K * = Bayesian Minimal Error Decision If P r (O K * / f ) = P r (O K / f ) MAX K

21 ASU MAT 591: Opportunities in Industry! Fingerprint Classification

22 ASU MAT 591: Opportunities in Industry! Fingerprint Basic Patterns

23 ASU MAT 591: Opportunities in Industry! Fingerprint Classification Process

24 ASU MAT 591: Opportunities in Industry! Prescreening Detection core delta core delta

25 ASU MAT 591: Opportunities in Industry! Prescreening Results

26 ASU MAT 591: Opportunities in Industry! Feature Measurements

27 ASU MAT 591: Opportunities in Industry! Local Area Classification

28 ASU MAT 591: Opportunities in Industry! Local Area Classification - continued

29 ASU MAT 591: Opportunities in Industry! Merge Results

30 ASU MAT 591: Opportunities in Industry! Decision Rules

31 ASU MAT 591: Opportunities in Industry! SPIE Medical Imaging 1998, San Diego CA 23 February 1998 Radiology Image Orientation Processing for Workstation Display Chung-Fu Chang, Kermit Hu, and Dennis L. Wilson

32 ASU MAT 591: Opportunities in Industry! Topics Introduction –Purpose –Nomenclature Approach –Chest front and side –Abdomen Results/Performance Conclusions

33 ASU MAT 591: Opportunities in Industry! Purpose Integrate into the Picture Archiving and Communication Systems (PACS) –Provide an end-to-end system to support telemedicine Enhance the productivity of radiology technicians –Reduce hospital operating cost

34 ASU MAT 591: Opportunities in Industry! Radiology Image Orientation Processing (RIOP) Automatically verify the body types, and identify image orientation Output image orientation –Chest images - two-letter code –Abdominal images - one-letter code

35 ASU MAT 591: Opportunities in Industry! Patient Orientation for Chest Front Images LHRHLFRF FLFRHLHR

36 ASU MAT 591: Opportunities in Industry! Patient Orientation for Chest Side Images PHAHPFAF

37 ASU MAT 591: Opportunities in Industry! Patient Orientation for Abdominal Images HF

38 ASU MAT 591: Opportunities in Industry! Approach Decision tree classifier Features measured from original or segmented images Bayesian Neural Network will be applicable, if the available data become adequate

39 ASU MAT 591: Opportunities in Industry! Chest Image Spatial-variant image segmentation Front (PA)Side (Lateral) Chest front or side test Side image orientation determination Front image orientation determination Lateral NotationPA Notation Unknown Read digital image

40 ASU MAT 591: Opportunities in Industry! Spatial-Variant Image Segmentation TiTi Histogram For each sub-region, compute a threshold for image segmentation

41 ASU MAT 591: Opportunities in Industry! Find side with maximum occupancy of image Y Y N N Unknown Occupancy >65% Y N Detect head; 1 and only 1 Detect lungs Rotate image Dynamically Segment Image Determine where heart is LH,RH,LF,RF, HL,HR,FL,FR Chest Front Image

42 ASU MAT 591: Opportunities in Industry! Chest Front Image Spatial-Variant Segmented Image Desired Orientation Original

43 ASU MAT 591: Opportunities in Industry! Dynamic Image Segmentation Threshold increases to ensure good definition of inner boundary of 2 lungs Lockheed Martin Proprietary

44 ASU MAT 591: Opportunities in Industry! Chest Side Image If patient orientation can be determined N Boundaries of side image Neck detection Calculate slope from neck up or down UnknownSegmented image Y Variation test PF, AF, PH, AH, HA, HP, FA or FP If patient orientation can be determined Y N PF, AF, PH, AH, HA, HP, FA or FP

45 ASU MAT 591: Opportunities in Industry! Chest Side Image Original Image Spatial-Variant Segmented Image

46 ASU MAT 591: Opportunities in Industry! Abdominal Image Spine Detection Sail up Sail down Image segmentation Pelvis Detection # of pelvis sections detected Pelvis sections detected more in upper portion Y H N F Y Y N Unknown Y Y N N Both are N’s # of pelvis sections detected Pelvis Detection

47 ASU MAT 591: Opportunities in Industry! Abdominal Image Original Detections of PelvisDesired Orientation

48 ASU MAT 591: Opportunities in Industry! Blind Test of RIOP Total of 445 images were collected for this test 256 chest images –185 chest front images –71 chest side images 23 abdominal images

49 ASU MAT 591: Opportunities in Industry! Chest Front vs Side Classification Algorithm Output Truth Front Side FrontSide 183 (98.9%) 2* 6**65 (91.5%) Total *poor image quality and peculiar image content **3 horizontal image, one half image

50 ASU MAT 591: Opportunities in Industry! Chest Front Orientation FL FR UNKHL 1 HRFLFRLHLFRHRFTotal HL HR LH LF RH RF Algorithm Output Truth

51 ASU MAT 591: Opportunities in Industry! Chest Front Orientation continued 164 out of 183, 89.6% were correctly oriented 180 out of 183, 98.4%, had foot of the image correctly identified 167 out of 183, 91.3% correctly identified handedness (left/right) of the patient

52 ASU MAT 591: Opportunities in Industry! 54 (96.4) 2 18 (88.9) 56 9 Algorithm Output Truth PF AF PFAFTotal Chest Side Orientation

53 ASU MAT 591: Opportunities in Industry! Chest Side Orientation continued 62 out of 65, 95.4% were correctly oriented When two images improperly classified as front were included in the side orientation test, these images were correctly identified as PF images –Percentage improves to 95.5%, 64 out of 67

54 ASU MAT 591: Opportunities in Industry! Algorithm Output Truth F H FHTotal Abdominal Image Orientation 22 (90.9) 2 01 (100.0) 22 1

55 ASU MAT 591: Opportunities in Industry! Conclusion Analyses indicated RIOP is very promising to be operational –RIOP identifies chest front from chest side almost perfectly –RIOP identified foot of the image at a high percentage, 98.4%  More data to further refine algorithm  Declare heart is always on right side of image, when foot of the patient is at bottom of image –Only verification of abdominal image is necessary, very few up- side-down abdominal images  Radiologists would have images upright for all but one or two exams per day RIOP will improve RT’s productivity and save hospital operating cost

56 ASU MAT 591: Opportunities in Industry! References [1] E.E. Hilbert and C-F Chang (1994) “Bayesian Neural Network ATR for multifeature SAR.” Proceedings of the SPIE’s International Symposium on Optical Engineering in Aerospace Sensing, Orlando, FL, April 4-9, : [2] Fingerprint Classification System, C-F Chang and E.E. Hilbert, U.S.A. patent number 5,572,597 (November 5, 1996). [3] Method for Orientating Electronic Medical Images, C-F Chang, K. Piggot and K. Hu, U.S.A. patent number 6,055,326 (April 25, 2000).

57 ASU MAT 591: Opportunities in Industry! Discussion

58 ASU MAT 591: Opportunities in Industry! Back-up Charts

59 ASU MAT 591: Opportunities in Industry! Original Spatial-Variant Segmented Image Desired Orientation Chest Front Image

60 ASU MAT 591: Opportunities in Industry! Chest Front Image Original Spatial-Variant Segmented Image Desired Orientation

61 ASU MAT 591: Opportunities in Industry! Chest Front Image Original Spatial-Variant Segmented Image Desired Orientation

62 ASU MAT 591: Opportunities in Industry! Dynamic Image Segmentation Threshold increases to ensure good definition of inner boundary of 2 lungs Lockheed Martin Proprietary