1. AUTOMATIC TARGET RECOGNITION AND DATA FUSION March 9 th, 2004 Bala Lakshminarayanan

2. Bala Lakshminarayanan, 9th March2 Outline Introduction Distributed processing DSN topologies Data fusion SFTB project Classification results

3. Bala Lakshminarayanan, 9th March3 Introduction Automatic Target Recognition Classify civilian targets with high accuracy 7 targets 3 sensors (IR, Grayscale, Acoustic) 3 nodes 3 scenarios Nodes are placed along the road

4. Bala Lakshminarayanan, 9th March4 Processing paradigms…(1) Centralized processing – fusion center High communication bandwidth Higher network cost Non-optimal processing, esp. when sensor coverage does not overlap Central node dependence

5. Bala Lakshminarayanan, 9th March5 Processing paradigms…(2) Distributed processing Redundancy – accurate classification Lesser network cost Reduced bandwidth requirement though increased communication between nodes Better response to rapid changes Needs proper architecture Energy efficiency

6. Bala Lakshminarayanan, 9th March6 Serial topology Event (H) Sensor 1Sensor 2Sensor n y1y1 y2y2 ynyn u1u1 u2u2 unun y i : Local observation u i : Local decision variable n : Number of sensors u 0 : Global decision variable

7. Bala Lakshminarayanan, 9th March7 Parallel topology Event (H) Sensor 1Sensor 2Sensor n y1y1 y2y2 ynyn u1u1 u2u2 unun Classifier Selection model Each classifier is an “expert” For feature x, classifier in its vicinity is given highest credit

8. Bala Lakshminarayanan, 9th March8 Parallel topology Event (H) Sensor 1Sensor 2Sensor n y1y1 y2y2 ynyn u1u1 u2u2 unun Fusion Center u0u0 Classifier Fusion model All classifiers trained over entire feature space Competitive model

9. Bala Lakshminarayanan, 9th March9 Data fusion…(1) Disparate sensors used for data collection Need to integrate results – fuse sensor data to give user ability to decide better Objective of fusion is to give one reliable, robust decision rather than many uncertain decisions Fusion levels Temporal Multi-modal Multi-sensor (from different nodes)

10. Bala Lakshminarayanan, 9th March10 Data fusion…(2) Temporal fusion Independent frames Majority voting Multi-modality fusion Different sensing modalities, all exposed to whole feature space Competitive rather than complementary BKS algorithm Multi-sensor fusion Handles faulty sensors MRI algorithm

11. Bala Lakshminarayanan, 9th March11 Data fusion…(3) Multi sensor fusion (From different nodes) Multi-modality fusion (From different sensing modalities) Temporal fusion (From different frames) Temporal fusion (From different frames) Multi-modality fusion (From different sensing modalities) Temporal fusion (From different frames) IR NODE (Frame1, frame2, Frame3….frameN) Temporal fusion (From different frames) GRAYSCALE NODE (Frame1, frame2, Frame3….frameN)

12. Bala Lakshminarayanan, 9th March12 BKS Classification…(1) Behaviour Knowledge Space Aggregates results obtained from individual classifiers Statistically, gives the optimal result OCR on 46,451 numerals shows BKS outperforms voting, Bayesian and Dempster- Shafer These approaches require the independence assumption – not so in real applications

13. Bala Lakshminarayanan, 9th March13 BKS Classification…(2) Independence assumption All classifiers are assumed to be equal Information for fusion is taken from confusion matrix of single classifier BKS avoids the independence assumption by concurrently recording decisions of all classifiers Behaviour of all classifiers recorded on a knowledge space - BKS

14. Bala Lakshminarayanan, 9th March14 BKS Classification…(3) Feature Vector Classifier Class Label Crisp Classifier, Fuzzy classifier, Possibilistic classifier Decision can he hardened using the maximum membership rule

15. Bala Lakshminarayanan, 9th March15 BKS Classification…(4) Majority voting Class labels are crisp or hardened Crisp label most represented is assigned to x Ties are broken randomly

16. Bala Lakshminarayanan, 9th March16 BKS Classification…(5) BKS s 1, s 2, …, s L are crisp labels assigned to x by classifiers D 1, D 2, …, D L respectively Every combination of labels is an index to an LUT

17. Bala Lakshminarayanan, 9th March17 BKS Classification…(6) Example of BKS c = 3, L = 2, N = 100 s 1, s 2 Number from each class Label 1,110/3/31 1,23/0/63 1,35/4/51,3 2,10/0/00 2,21/16/62 2,34/4/41,2,3 3,17/2/41 3,20/2/53 3,30/0/63

18. Bala Lakshminarayanan, 9th March18 SFTB Framework Start Grab frames from dataset Continuous, non continuous Segment using bgSubtract() Background image Extract features invMoment() Normalize, write database files Classify readData(), knn() End Inputs-nodeID, scenario…

19. Bala Lakshminarayanan, 9th March19 Results…(1) 30 frames from each node – 5 testing, 25 training 7 targets 3 scenarios 2 nodes (IR, Grayscale) 6 databases with 210 feature vectors 2 versions – database1, database2

20. Bala Lakshminarayanan, 9th March20 Results…(2) Node px12, Scenario 1, k=7, Classification accuracy 62.86% Targets1234567 14-----1 2-5----- 3--4--1- 4---5--- 5----1-4 6-22--1- 73-----2 1234567 13-1-1-- 2-5----- 31-4---- 4--221-- 5----4-1 611---3- 7-----23 Node in23, Scenario 1, k=7, Classification accuracy 62.86%

21. Bala Lakshminarayanan, 9th March21 Results…(3) Scenariok=7k=9k=11 162.8668.57 6 71.43 2588.57 80.00 Scenariok=7k=9k=11 162.8645.7151.43 657.1454.2857.14 2548.57 34.28 Node PX12 Node IN23

22. Bala Lakshminarayanan, 9th March22 Future work Perform fusion between IR and grayscale image data Perform fusion between images from different scenarios

23. Bala Lakshminarayanan, 9th March23 References X. Wang, H. Qi, S. S. Iyengar, “Collaborative multi-modality target classification in distributed sensor networks”, International Conference on Information Fusion (ICIF), July 2002 R. Viswanathan, P.K. Varshney, “Distributed Detection with multiple sensors: Part 1-Fundamentals”, Proceedings of the IEEE, Vol 85, Jan 1997 L.I. Kuncheva, J.C. Bezdek, Robert P.W. Duin, “Decision templates for multiple classifier fusion: an experimental comparison”, Pattern Recognition, Vol 34, 2001 Y.S. Huang, C.Y. Suen, “A method for combining multiple experts for the recognition of unconstrained handwritten numerals”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 17, Jan 1995