1. Intelligent Sensor Network Signal and Information Processing For LTER Applications Yu Hen Hu University of Wisconsin – Madison Dept. Electrical and Computer Engineering Madison, WI 53706 hu@engr.wisc.edu Presented at LTER ASM 03, 9/20/2003

2. © 2003 by Yu Hen Hu 2 Outline Long term environmental monitoring requirements Intelligent signal and information processing (ISIP) –Intelligent agent and ISIP –Needs for intelligence Enhance performance Reduce cost Work in progress –Detection of changes –Clustering of events –Sampling frequency determination Future works –Intelligent monitoring –Intelligent maintenance –Intelligent data analysis

3. © 2003 by Yu Hen Hu 3 Challenges in Long Term Environmental Monitor Objectives and hypothesis –Specific biological, environmental phenomena to be observed and analyzed –Formulate hypothesis based on existing data Experiment design –Location, duration of observation, –likelihood of onset of events of interests –Instrumentation Types of sensors, Deployment plan Maintenance plan –Data archiving plan Signal and information processing –On-line, real time control Turn on/off sensor Adjust sampling frequency Data routing/collecting Streaming of video/image on-demand Calibration and self- monitoring –Off-line data archival, retrieval Analysis, visualization inference

4. © 2003 by Yu Hen Hu 4 Intelligent Agent –Intelligent agent are persistent software/hardware systems that perceive, reason, act, and communicate on behalf of human users Sensor action knowledge Environment Agent

5. © 2003 by Yu Hen Hu 5 Characteristics of an IA Autonomous execution Goal seaking Persistent within or as a part of a system Able to reason during action selection Acting for another with authority granted by another Interact with other agents or human via dialog or some agent communication language

6. © 2003 by Yu Hen Hu 6 ISIP: Intelligent Signal and Information Processing Signal processing –The sampling, conditioning, compression, transmission, and analysis of numerical measurements of the environment based on sensor readings Information processing –The handling of non-numerical data to coordinate collaboration, and control operations ISIP –Perform signal and information tasks using intelligent agent –Tasks: statistical and heuristic reasoning, including hypothesis testing, classification, estimation, data fusion, etc. –Tools: neural network, expert system, fuzzy logic, genetic algorithm, pattern classifiers, time series analysis, statistical learning, support vector machine, Bayesian network, planning, etc.

7. © 2003 by Yu Hen Hu 7 Wisconsin Long-Term Ecological Research (LTER) project Three Buoy Projects 1) Sparkling raft Serial communication Fixed buoy location Simple data format that must conform to historical format Wireless takes the place of serial cable, but download timing automated 2) 3 small roving buoys Serial communication Flexible buoy locations Complex data format with intensive post-processing Wireless simply takes the place of a serial cable 3) Large profiling buoy Bidirectional ethernet communication Fixed buoy location Simple but flexible data format Real-time data with web publishing and buoy control http://www.limnology.wisc.edu/

8. © 2003 by Yu Hen Hu 8 Wisconsin Limnology Lab Buoy Wireless Sensor Network

9. © 2003 by Yu Hen Hu 9 How Buoy Data Are Processed?

10. © 2003 by Yu Hen Hu 10 What a Buoy Will Do?

11. © 2003 by Yu Hen Hu 11 Water Temperature Data

12. © 2003 by Yu Hen Hu 12 A Change Detection Problem

13. © 2003 by Yu Hen Hu 13 Sensor Node Signal Processing Requirements: –Simple algorithm –Robust performance –Low power operation Trend removal –Use linear phase FIR filter Outlier detection –Simple statistical method to detect outliers

14. © 2003 by Yu Hen Hu 14 Decision Fusion

15. © 2003 by Yu Hen Hu 15 Enabling Technologies Enabling information technologies –Internet networking and ad hoc mobile network –Wireless communication –Micro-electronic mechanical (MEM) devices –System-on-chip technology integrating Analog + digital Sensing + wireless communication + processing + actuation

16. © 2003 by Yu Hen Hu 16 Technical Challenges Self-configuration, –Self-organization –Self maintenance –Services discovery –Directory services Collaborative sensor signal processing –Sampling –Encoding, compression –aggregation –Event detection –Target identification, situation awareness –Tracking Secure operation –Privacy protection Ensure sensor information is accessed only by authorized personnel –Fault tolerance, high availability –Safety Non-intrusive, Safe actuation –Sabotage resistance, security

17. © 2003 by Yu Hen Hu 17 Self-Configuration Purpose –Reduce deployment and configuration cost Vision –Sensors are deployed randomly (ad hoc network) to reach a desired local density –After deployment, sensors periodically communicate to each other to establish and maintain a connected network. –Directory (configuration information) will be aggregated, and published to authorized agent. –Sensors monitor network status and periodically report to an external monitoring agent –Sensor network re-organize itself in case Its mission is changed as directed from an authorized external agent Traffic/load changed over time Sensors’ physical position changed if they are mobile Part of the network malfunction due to sensor failure, or communication failure

18. © 2003 by Yu Hen Hu 18 Self Configuration How to join a physical network; that is, how is it authorized and given a network address and a network identity? Once an entity is on the network and wishes to provide a service to other entities on the network, how does it indicate that willingness? If an entity is looking for a service on the network, how does it go about finding that service? How does geographic location affect the services an entity can discover or select for use?

19. © 2003 by Yu Hen Hu 19 Collaboration Sensors collaborate to achieve network-wide processing objective: –Higher performance –Lower resource (energy, band-width) consumption Challenge: How to achieve globally optimal results –Using local, distributed criteria –With local communication –Using minimum amount of energy Approach –Exploring redundancy and correlation Densely deployed sensor field Sensor readings are correlated

20. © 2003 by Yu Hen Hu 20 Collaborative Sensor Signal Processing Sampling –How to use lowest sampling rate/density to achieve desired spatial- temporal accuracy? Compression –How to reduce overall sampled data that needs to be transmitted over wireless channel? Aggregation –How to summarize information from multiple sensor without overload wireless channel Event detection –How to deploy sensors so that a desired event can be detected with a specified accuracy? Target identification, situation awareness –How to classify targets when there are multiple targets present Tracking –How to coordinate sensor activities to track multiple target effectively.

21. © 2003 by Yu Hen Hu 21 Sampling Temporal resolution –How many samples per unit time? –Reduce rate to conserve energy, band-width –Based on underlying physics –Nyquist theorem for band- limited signals –Adaptive sampling rate for non-stationary signals Spatial resolution –For visual signals –How small the frame size can still allow Target detection Subject identification Tracking Other monitoring functions –Adaptive spatial resolution Higher resolution in region of interests Spatial-temporal sampling –Different camera/sensors coordinate to sample at lower rate while achieving higher resolution than a single camera

22. © 2003 by Yu Hen Hu 22 Compression Sensors closer to each other physically, may sample similar (correlated) data It waste energy and band- width to transmit data un- compressed. Exploiting the correlation of sensor data among adjacent sensors, amount of data transmitted can be further reduced. Example: compression of multiple video streams taken from neighboring cameras If sensor A reading = x, then sensor B reading = x  2, and vice versa Suppose both readings have range [0, 127] If reading A = 25, then sensor A knows sensor B’s reading in [23, 27]. If sensor B sends its reading in 7 bits, sensor A knows the receiving end must know reading A in [21, 29] Needs only 3-bits to encode! D. Slepian, and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Information Theory, vol. 19, 1973, pp. 471-480

23. © 2003 by Yu Hen Hu 23 Aggregation Problem Statement –Find an estimate of a statistic of sensor measurements of a group of sensors with minimum amount of wireless transmission Assume –Sensors can overhear neighboring sensor’s transmission –Sensor readings are correlated Example: –Find maximum reading among N sensors One possible protocol: –Sensor with higher reading report first. –Sensors with readings larger than reported readings will report with collision control –Sensors whose readings smaller than reported readings remain silent. –Wait for a pre-specified time period. The last reported reading is the maximum with high probability

24. © 2003 by Yu Hen Hu 24 Event/Target Detection Statistical hypothesis testing tasks Collaborative detection –Correctly detect an event or a target while minimizing cost (energy and band-width) –Individual sensors may not detect correctly due to Limited range, Limited scope noise Methods of collaborative detection –Decision fusion –Multi-modality detection Low cost sensor detect first with higher false alarm rate High cost sensor (visual sensors) to verify detection results Challenges –Not all sensors report detection –Not all reported detection will be forwarded to fusion center

25. © 2003 by Yu Hen Hu 25 Target classification/ Event awareness Pattern classification problems –Low power feature extraction –Decision fusion Feature extraction –Invariant to variations –Cheap to compute –Local to each sensor Decision fusion –Only discrete set of decisions needs to be transmitted over wireless channel Event awareness –Detecting a particular event such as traffic accident –Require understanding of a sequence of states using hidden Markov model –Requires detection of onset and offset of an event –Requires tracking of objects of interests

26. © 2003 by Yu Hen Hu 26 Conclusion Sensor network is a new application area for computer vision, graphics and image processing It requires multi-modality, multimedia processing under the constraint of minimizing communication and energy consumption.

27. © 2003 by Yu Hen Hu 27 Great Duck Island Monitoring Project Starting time: Spring 2002, Participants: –Intel Research Laboratory at Berkeley –the College of the Atlantic in Bar Harbor –University of California at Berkeley Task: –deploy wireless sensor networks on Great Duck Island, Maine. Mission: –monitor the microclimates in and around nesting burrows used by the Leach's Storm Petrel. Goal: –to develop a habitat monitoring kit that enables researchers worldwide to engage in the non-intrusive and non-disruptive monitoring of sensitive wildlife and habitats http://www.greatduckisland.net/

28. © 2003 by Yu Hen Hu 28 Habitat and the Bird Habitat to be monitored (up, yellow: microphone Red: camera) and the Leach’s storm petrel (right)

29. © 2003 by Yu Hen Hu 29 GDI Sensor Network Autonomous sensor nodes: "motes“placed in areas of scientific interest, form a multihop network (sensor patch) Each patch network gateway mote has an external directional antenna forward data to basestation Basestation: a laptop in the light house (350ft away) stores the data in a database and connect to Internet. Mainwaring, et. Al, wireless sensor network for habitat monitoring, ACM workshop on wireless Sensor networks and applications, sept. 2002, Atlanta, GA

30. © 2003 by Yu Hen Hu 30 Mica Sensor Node Left: Mica II sensor node 2.0x1.5x0.5 cu. In. Right: weather board with temperature, thermopile (passive IR), humidity, light, acclerometer sensors, connected to Mica II node Single channel, 916 Mhz radio for bi-directional radio @40kps 4MHz micro-controller 512KB flash RAM 2 AA batteries (~2.5Ah), DC boost converter (maintain voltage) Sensors are pre-calibrated (±1- 3%) and interchangeable

31. © 2003 by Yu Hen Hu 31 MICAII Operating Power Budget OperationnAh Transmitting a packet20.000 Receiving a packet8.000 Radio listening for 1 ms1.250 Operating sensor for 1 sample (analog)1.080 Operating sensor for 1 sample (digital)0.347 Reading a sample from ADC0.011 Flash read data1.111 Flash write/erase data83.333

32. © 2003 by Yu Hen Hu 32 GDI Nodes Low Power Strategies 9-month deployment period on 2 AA batteries  8.148 mAh/day Nodes nearing gateway consume more power to forward packages from leaf nodes Sleep mode: periodically wake up to listen to command Energy reduction measures: –Turning off sensor, radio, –Putting processor into deep sleep mode with reduced supply voltage –Bypass DC booster –Pull-up I/O pin voltage –Energy available for operation ~ 6.9 mAh/day

33. © 2003 by Yu Hen Hu 33 Sensed Data Raw thermopile data from GDI during 19-day period from 7/18- 8/5/2002. Show difference between ambient temperature and the object in the thermopile’s field of view. It indicates that the petrel left on 7/21, return on 7/23, and between 7/30 and 8/1