Achieving Better than Human Design in Detecting Events of Interest in Bandwidth Constrained Sensor Networks Kalyan Veeramachaneni, Lisa Osadciw Development and Research in Evolutionary Algorithms for Multi sensor Smart Networks (DREAMSNet) Department of Electrical Engineering and Computer Science Syracuse University, New York GECCO 2008, Human Competitive Results Awards, July 14, Atlanta, U.S.A

2 What are we detecting? Modern day society relies on detection or determining the meaning of the presence or absence of a signal Modern day society relies on detection or determining the meaning of the presence or absence of a signal Digital Communications Digital Communications Pipeline/Bridges crack detection Pipeline/Bridges crack detection Genuine User detection using biometrics Genuine User detection using biometrics Presence of aircraft, ships, or motor vehicles Presence of aircraft, ships, or motor vehicles Locating emergency personnel Locating emergency personnel Weather Phenomena Weather Phenomena Building Security Building Security Sensors are located in remote areas making decisions using a variety of criteria Sensors are located in remote areas making decisions using a variety of criteria Maximum A-Posteriori Criterion Maximum A-Posteriori Criterion Maximum Likelihood Criterion Maximum Likelihood Criterion Minimum Error Criterion Minimum Error Criterion

3 Bandwidth Constrained Detection Networks Sensor1 Sensor2 X1X1 X2X2 Fusion Rule u1u1 u2u2 Second Classifier Only OR AND First Sensor Only Likelihood density model for a sensor Noise only Event

4 Bandwidth Constrained Detection Networks Two types of Errors need to be reduced Two types of Errors need to be reduced If the entire observation value is transmitted to a central processing node, an efficient machine learning technique can be designed to achieve better accuracy If the entire observation value is transmitted to a central processing node, an efficient machine learning technique can be designed to achieve better accuracy Shown below are samples of observations, belong to events, to noise. Shown below are samples of observations, belong to events, to noise. 9 to 32 bits required per sample if all bits are transmitted 9 to 32 bits required per sample if all bits are transmitted Reduces to 1 bit decision if decision is transmitted instead Reduces to 1 bit decision if decision is transmitted instead Misses: Fail to detect an event False Alarms: detecting an event that did not occur Threshold on Sensor 1 Threshold on Sensor 2 Event is declared only in this quadrant, i.e. AND rule Noise * Event

5 (E)Competitive Result: Correlated Sensors Designs for 0.1 Correlation LRT (Human) Based Design: 2 thresholds on each sensor 2 Sensor only fusion rule Region where an event is declared PSO Based Design: Simple 1 Threshold for each sensor AND fusion rule Very few errors Region where an event is declared

6 Humies Categories Covered (G) The result solves a problem of indisputable difficulty in its field. (G) The result solves a problem of indisputable difficulty in its field. (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions.

7 HUMIES Category G The result solves a problem of indisputable difficulty in its field. The result solves a problem of indisputable difficulty in its field. Amount of Research and Publications on Topic Indicates Complexity Amount of Research and Publications on Topic Indicates Complexity Quick Check Research Publications Quick Check Research Publications 120 Journal Articles with Approximately 45 Discussing Similar Design Issues 120 Journal Articles with Approximately 45 Discussing Similar Design Issues 48 Textbooks At Least Currently On Sale In This Area 48 Textbooks At Least Currently On Sale In This Area 5 Dissertations deal with same problem and provide human developed designs 5 Dissertations deal with same problem and provide human developed designs Paper Published that Addresses the Difficulty Paper Published that Addresses the Difficulty John N Tsitsiklis, Michael Athans, “On Complexity of Decentralized Decision making and detection problems” 23rd IEEE Conference on Decision and Control, 1984 John N Tsitsiklis, Michael Athans, “On Complexity of Decentralized Decision making and detection problems” 23rd IEEE Conference on Decision and Control, 1984 Optimizing Distributed Detection for 2 Sensors Optimizing Distributed Detection for 2 Sensors Independent sensors: Intractable Independent sensors: Intractable Correlated sensors: NP Complete - Correlated sensors: NP Complete -

8 HUMIES Category B The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal Much Research Published in Area Since the 50s/60s Beginning in Radar Much Research Published in Area Since the 50s/60s Beginning in Radar Type I/Type II Errors Type I/Type II Errors Fail to detect the event Fail to detect the event Detect an Event that did not occur Detect an Event that did not occur Decouple the two problems: optimize thresholds and design best fusion rule separately Decouple the two problems: optimize thresholds and design best fusion rule separately When only labeled training datasets are available performance is sensitive to threshold search precision When only labeled training datasets are available performance is sensitive to threshold search precision When likelihood models are available Optimize Threshold Exceed Ratio of Conditional Distributions

9 Sensor1 Sensor2 X1X1 X2X2 Fusion Rule u1u1 u2u2 Human Design Solution: Likelihood Ratio Test (LRT) Design Optimize thresholds individually by keeping other thresholds and fusion rule constant Use LRT for independent or correlated deriving fusion rule Human Design Solution: Person-by-Person Optimal (PBPO) for Independent Sensors Human Competitive Result: Particle Swarm Optimization (PSO) Based Design Joint optimization of thresholds and Fusion Rule No closed form solution exists

10 HUMIES Category B (cont) Particle Swarm Optimized Detector Simplifies Sensor Network Adaptation Particle Swarm Optimized Detector Simplifies Sensor Network Adaptation Able to combine performance parameters to simultaneously handle a variety of situations Able to combine performance parameters to simultaneously handle a variety of situations consider resources (energy and communication bandwidth) consider resources (energy and communication bandwidth) Reduce type I or type II errors across different degrees of correlation Reduce type I or type II errors across different degrees of correlation Simpler Receiver Simpler Receiver Single threshold design compared to LRT based designs that can lead to multiple thresholds as the Likelihood ratio becomes non linear Single threshold design compared to LRT based designs that can lead to multiple thresholds as the Likelihood ratio becomes non linear Adapt to design for either probability density models or a labeled training dataset provided Adapt to design for either probability density models or a labeled training dataset provided Automatically Handles the Heterogeneity in Practical Sensor Networks Automatically Handles the Heterogeneity in Practical Sensor Networks

11 HUMIES Category D The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. 5 Papers Published Including a Best Paper 5 Papers Published Including a Best Paper Correlated Sensors Correlated Sensors Kalyan Veeramachaneni and Lisa Osadciw, “Design of Distributed Detection Systems with Heterogeneous Correlated Sensors," 44th Annual Allerton Conference on Communications and Control, Allerton Park, Illinois, September, Kalyan Veeramachaneni and Lisa Osadciw, “Design of Distributed Detection Systems with Heterogeneous Correlated Sensors," 44th Annual Allerton Conference on Communications and Control, Allerton Park, Illinois, September, Independent Sensors Independent Sensors Kalyan Veeramachaneni, Lisa Osadciw, Pramod Varshney“Adaptive Multimodal Biometric Management Algorithm,” IEEE Transactions on Systems Man and Cybernatics : Part C: Applications and Reviews, Vol. 35, No. 3 August Kalyan Veeramachaneni, Lisa Osadciw, Pramod Varshney“Adaptive Multimodal Biometric Management Algorithm,” IEEE Transactions on Systems Man and Cybernatics : Part C: Applications and Reviews, Vol. 35, No. 3 August Applications Applications Biometrics: Kalyan Veeramachaneni, Nisha Srinivas, Lisa Osadciw, and Arun Ross, “Designing Optimal Fusion Strategies for Correlated Biometric Classifiers”, IEEE CVPR Conference, Anchorage, Alaska, June, (Best Paper Award) Biometrics: Kalyan Veeramachaneni, Nisha Srinivas, Lisa Osadciw, and Arun Ross, “Designing Optimal Fusion Strategies for Correlated Biometric Classifiers”, IEEE CVPR Conference, Anchorage, Alaska, June, (Best Paper Award) Pipeline Crack Detection: Kalyan Veeramachaneni, Weizhong Yan, Kai Goebel, and Lisa Osadciw, “Improving Classifier Fusion Using Particle Swarm Optimization”, IEEE Multi-Criteria Decision Making (MCDM) Symposium, Honolulu, Hawaii, April, Pipeline Crack Detection: Kalyan Veeramachaneni, Weizhong Yan, Kai Goebel, and Lisa Osadciw, “Improving Classifier Fusion Using Particle Swarm Optimization”, IEEE Multi-Criteria Decision Making (MCDM) Symposium, Honolulu, Hawaii, April, Adaptive Sensor Management Adaptive Sensor Management Lisa Osadciw and Kalyan Veeramachaneni, “Sensor Management through Efficient Fitness Function Design," Proceedings of 41st Annual Asilomar Conference on Signals, Systems, and Computers, Asilomar, CA, November, Lisa Osadciw and Kalyan Veeramachaneni, “Sensor Management through Efficient Fitness Function Design," Proceedings of 41st Annual Asilomar Conference on Signals, Systems, and Computers, Asilomar, CA, November, 2007.

12 HUMIES Category E The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. Long-standing problem since the 1950s in Radar Research Long-standing problem since the 1950s in Radar Research Succession of better solutions as discussed in Category B Succession of better solutions as discussed in Category B First Single Detectors Derived for the Following Criterion First Single Detectors Derived for the Following Criterion Maximum A-Posteriori Criterion – maximize the posteriori probability of belonging to one event to the other possible event Maximum A-Posteriori Criterion – maximize the posteriori probability of belonging to one event to the other possible event Maximum Likelihood Criterion – maximizes probability of belonging (likelihood) to event Maximum Likelihood Criterion – maximizes probability of belonging (likelihood) to event Minimum Error Criterion – minimize the number of errors in decisions Minimum Error Criterion – minimize the number of errors in decisions

13 HUMIES Category E (cont): Matched Filter Designed in 50s and 60s from Radar Matched Filter Designed in 50s and 60s from Radar Maximum Signal to Noise Criterion – maximize signal over the noise background to assist detection by matched filter (North, Van Vleck, Middleton) Maximum Signal to Noise Criterion – maximize signal over the noise background to assist detection by matched filter (North, Van Vleck, Middleton) Inverse Probability Criterion – (Wald, Neyman, Pearson) Inverse Probability Criterion – (Wald, Neyman, Pearson) Likelihood Ratio - based on Shannon’s information theory (Woodward & Davis) Likelihood Ratio - based on Shannon’s information theory (Woodward & Davis) Distributed Detection (Tenney & Sandell-1979 through today) Distributed Detection (Tenney & Sandell-1979 through today) Chair Z., P. K. Varshney, "Optimal Data Fusion in Multiple Sensor Detection Systems," IEEE Trans. on Aerospace and Elect. Systems, Vol. AES-22, No. 1, pp , Jan Chair Z., P. K. Varshney, "Optimal Data Fusion in Multiple Sensor Detection Systems," IEEE Trans. on Aerospace and Elect. Systems, Vol. AES-22, No. 1, pp , Jan Tang, Z. -B., K. R. Pattipati, and D. L. Kleinman, "An Algorithm for Determining the Decision Thresholds in a Distributed Detection Problem," IEEE Trans. on Systems, Man, and Cybernetics, Vol. SMC-21, pp , Jan./Feb Tang, Z. -B., K. R. Pattipati, and D. L. Kleinman, "An Algorithm for Determining the Decision Thresholds in a Distributed Detection Problem," IEEE Trans. on Systems, Man, and Cybernetics, Vol. SMC-21, pp , Jan./Feb Kam., M., Q. Zhu., and W. S. Gray, "Optimal Data Fusion of Correlated Local Decisions in Multiple Sensor Detection Systems," IEEE Transactions on Aerospace and Elect. Syst., Vol. 28, pp , July Kam., M., Q. Zhu., and W. S. Gray, "Optimal Data Fusion of Correlated Local Decisions in Multiple Sensor Detection Systems," IEEE Transactions on Aerospace and Elect. Syst., Vol. 28, pp , July Peter Willet, Peter F. Swaszek, Rick S. Blum, "The Good, Bad, and Ugly : Distributed Detection of Known Signal in Dependent Gaussian Noise," IEEE Transactions on Signal Processing, Vol. 48, No. 12, December Peter Willet, Peter F. Swaszek, Rick S. Blum, "The Good, Bad, and Ugly : Distributed Detection of Known Signal in Dependent Gaussian Noise," IEEE Transactions on Signal Processing, Vol. 48, No. 12, December Kalyan Veeramachaneni, Lisa Ann Osadciw, Pramod K Varshney, "An Adaptive Multimodal Biometric Fusion Algorithm Using Particle Swarm", Proceedings of SPIE, Aerosense, April 21-25, 2003, Orlando. Kalyan Veeramachaneni, Lisa Ann Osadciw, Pramod K Varshney, "An Adaptive Multimodal Biometric Fusion Algorithm Using Particle Swarm", Proceedings of SPIE, Aerosense, April 21-25, 2003, Orlando. Saeed A. Aldosari, Jose M. F. Moura, “Fusion in Sensor Networks with Communication Constraints”, International Symposium on Information Processing in Sensor Networks, April 26-27, 2004, Berkeley, CA. Saeed A. Aldosari, Jose M. F. Moura, “Fusion in Sensor Networks with Communication Constraints”, International Symposium on Information Processing in Sensor Networks, April 26-27, 2004, Berkeley, CA.

14 HUMIES Category E (cont): Swarm Solution: Type I/Type II Errors Type I/Type II Errors Errors are Balanced in Real Time Based on Current System Needs Errors are Balanced in Real Time Based on Current System Needs Simultaneously reduce, “Failure to detect the event”, “Detect an Event that did not occur ” Simultaneously reduce, “Failure to detect the event”, “Detect an Event that did not occur ” Reduce Communication Bandwidth Reduce Communication Bandwidth Decisions at Sensor to Reduce Message Size Saving Bandwidth Decisions at Sensor to Reduce Message Size Saving Bandwidth Fusion Architecture Can Be Modified in Real-Time Based on Bandwidth and Energy Needs Fusion Architecture Can Be Modified in Real-Time Based on Bandwidth and Energy Needs Minimize Energy Minimize Energy Save in communications with Smaller Messages and Fewer Through Fusion Save in communications with Smaller Messages and Fewer Through Fusion Reduce computations with Simpler designs and Fusion Rules Reduce computations with Simpler designs and Fusion Rules Ease of Adaptation to Other Applications Ease of Adaptation to Other Applications Communication Management for Any Wireless Sensor Network and Architecture Communication Management for Any Wireless Sensor Network and Architecture Various Sensor Networks for Aircraft Routing at Airports, First Response Networks, Large, Remote Sensor Networks, Health Monitoring Sensor Networks, and etc. Various Sensor Networks for Aircraft Routing at Airports, First Response Networks, Large, Remote Sensor Networks, Health Monitoring Sensor Networks, and etc.

15 (E) Competitive Result: Independent Sensors Number of Sensors PBPO PSO PSO % Improvements in accuracy % Improvements in accuracy e e e e Human Design Accuracy PSO Resulting Accuracy PBPO-Person-By-Person Optimal PSO – Particle Swarm Optimization

16 (E)Competitive Result: Correlated Sensors Human Design 54% 13% 2.5%

17 Humies Categories In Summary (G) The result solves a problem of indisputable difficulty in its field. (G) The result solves a problem of indisputable difficulty in its field. This is an NP complete problem which also becomes too complex as the sensors become dependent. This is an NP complete problem which also becomes too complex as the sensors become dependent. (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer- reviewed scientific journal (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer- reviewed scientific journal Problem Studied Since 1950s with Suboptimal Solutions Problem Studied Since 1950s with Suboptimal Solutions PSO Allows the Coupled Threshold – Fusion Rule Problem to Remain Coupled PSO Allows the Coupled Threshold – Fusion Rule Problem to Remain Coupled PSO is able to solve the problem for different problem types PSO is able to solve the problem for different problem types (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. We have published 5 papers including 1 that recently received “Best Paper” in an application domain We have published 5 papers including 1 that recently received “Best Paper” in an application domain (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. As the network grows, the PSO performance also grows w.r.t. the human-created solution. As the network grows, the PSO performance also grows w.r.t. the human-created solution.

18 Why this is the best? Significance of the Impact on a Wide Range of Applications Significance of the Impact on a Wide Range of Applications - Military - Health Monitoring - Homeland Security - Environmental Monitoring - Smart and Safe Buildings - Vehicle Health Monitoring Ease of Adapting Solution to the Complexities of Practical Problems Ease of Adapting Solution to the Complexities of Practical Problems - Imperfect Propagation Channels - Multi-User Interference - Changing Architectures - Resource Constraints Solves Distributed Detection Problems Too Complex in Past Solves Distributed Detection Problems Too Complex in Past - Multiple Imperfections - Complex Sensor Architectures - Complex Interference Environments

19 Thank you! Thank you!

20 Sensor1 Sensor2 X1X1 X2X2 Fusion Rule u1u1 u2u2 Human Design Solution: Likelihood Ratio Test (LRT) Design LRT based fusion rule for independent sensors LRT based Fusion Rule for correlated sensors

21 (E)Competitive Result: Correlated Sensors Designs for 0.9 Correlation LRT (Human) Based Design: 2 thresholds on each sensor 2 Sensor only fusion rule Region where an event is declared PSO Based Design: Simple 1 Threshold for each sensor AND fusion rule Higher number of errors, but still better Region where an event is declared