1. Fault Tolerant Sensor Network for Border Activity Detection B. Cukic, V. Kulathumani, A. Ross Lane Department of CSEE West Virginia University NC-BSI, December 2008

2. NC – BSI 2008 2 Problem Statement Develop methods that supplement wireless camera sensor networks with communication and analysis algorithms that succinctly interpret the data yielding intelligent surveillance. Methodology –On-board scene analysis for reduced communication. –Fusion of frames from cameras. –Collaborative detection and human identification for improved coverage, robustness, resilience.

3. NC – BSI 2008 3 3 Motivating scenarios Network of image sensors + biometric recognition –Building block for surveillance application Scenario: Airport security

4. NC – BSI 2008 4 4 Prior work Network of cameras + biometric systems Building mounted camera Suspicious car – person exiting Demonstrated at Ohio State Nov 2007 Scenario 2 : Surveillance Communicates with In building doorway camera Indoor camera takes picture

5. NC – BSI 2008 5 5 Problems Collaborative detection of an event of interest and simultaneous human (face) recognition in a dynamic scene Coordinate image acquisition –To capture partial views that maximize biometric content. –Acquire partial snapshots and fuse them together to reconstruct full image. Develop algorithms that succinctly interpret captured data to detect human activity, establish identity and analyze intent.

6. NC – BSI 2008 6 6 Work Elsewhere Previous work in the literature focused on –Design of efficient retrieval mechanisms of extensive video data to a central location –Storage of compressed data in-network to be used on the basis of queries from users Previous work on camera coverage: –Calculating overlap of coverage in an existing placement to obtain duty cycling and energy efficiency –Calculate optimal placement of cameras to track objects in horizontal plane, treating them as point objects

7. NC – BSI 2008 7 7 Our Approach Do not continuously capture data –Data overload –Network congestion –Hard to identify and focus on event of interest Cameras exchange metadata to collaboratively focus on object of interest Multiple cameras capture images –How much processing can be done on-board each camera? Choosing the “most informative” frame? Matching against a short watch list? –Collaborative identity determination. –Increased system resilience (fault tolerance).

8. NC – BSI 2008 8 8 Research Plan Integration of cameras with wireless radios, pocket PC and camera control –Sencor board: IMB Multimedia Board (video, audio…) –Processor board: IMOTE2 boards (integrated radio, XScale® Processor, 256kB SRAM, 32MB FLASH, 32MB SDRAM ) –Integrate with PDA devices + wireless (802.11 / 802.15.4) –Camera control protocols Positioning algorithm for optimal placement of cameras to maximize biometric content

9. NC – BSI 2008 9 Leverage The Center for Identification Technology Research (NSF I/UCRC) Ascertaining Identity within Human Networks in Night Environments (DoD – Office of Naval Research)

10. NC – BSI 2008 10 Deliverables Year 1: –Laboratory set-up, –Face detection and tracking algorithms, –Initial fusion algorithms, –On-board processing. Years 2-5: –Hand-off protocols –Performance and reliability studies with fusion –Proof-of-concept demonstration –Robustness analysis of identification –Group intent applications.