Suspicious Behavior in Outdoor Video Analysis - Challenges & Complexities Air Force Institute of Technology/ROME Air Force Research Lab Unclassified IED test sequences showing dropped package from vehicle (DPV) Combination of motion analysis and change detection –Homogenous regions and aperture problem and for optic flow approaches –Learning appropriate background for change (ghost objects appear due to slow or fast learning) –Global camera motion/jitter Occlusion and Camouflage Environmental problems –Dust and smoke –Wind –local object motion (swaying of branches, shadows) –Precipitation –rain, slow etc. –Clutter (background model) Illumination problems –Shadows (static and moving cast shadow) - missed objects or false detections –Glare – false detections, object shape and trajectory distortions –Sudden illumination changes (cloud movements) – false detections –Low contrast or color saturation

Dropped package from vehicle (DPV) sequences - Logitech Orbit 20ft Vertical Run 2 Frame 150: Event of interest marked

Detecting Occluded Event Sequence - Logitech Orbit 20ft Vertical Run 3 Frame 121: Event of interest marked

Glare and Shadows Sequence - Logitech Orbit 10ft Vertical Run 2 Frame 43: False detection due to glareFrame 141: Event of interest marked

Dust and Shadows Sequence - Logitech Orbit 10ft Vertical Run 3 Frames 72&170: False detection due to dust Frame 159: Event of interest marked

Dust and Shadows Sequence - Logitech QuickCamPro ft Vertical Run 3 Frame 150: Event of interest missed due to shadow and insufficient contrast Frame 91: False detections due to dust

Sequence - Logitech QuickCamPro ft Vertical Run 2 No event of interest

Sequence - Logitech QuickCamPro ft Vertical Run 3 Frame 104: Event of interest marked

Effect of Learning Rate in Background Modeling Sequence - Logitech Orbit 10ft Vertical Run 2 Frame 86: Correct Detection when in motion Frame 134: Object that stops for a while blends into the background Frame 210: Ghost object left behind (due to slow background learning using Mixture of Gaussians) when the car starts to move again

Problems with Flow-based Approaches Sequence - Logitech QuickCamPro ft Vertical Run 1 Frame 24: Aperture problem, motion of homogeneous regions is not detected Frame 35: Non-moving objects not detected Frame 83: Larger temporal window results in false detections and larger object boundaries

Suspicious Behavior in Outdoor Video Analysis - Challenges & Complexities Combination of motion analysis and change detection –Homogenous regions and aperture problem and for optic flow approaches –Learning appropriate background for change (ghost objects appear due to slow or fast learning) –Global camera motion/jitter Occlusion and Camouflage Environmental problems –Dust and smoke –Wind –local object motion (swaying of branches, shadows) –Precipitation –rain, slow etc. –Clutter (background model) Illumination problems –Shadows (static and moving cast shadow) - missed objects or false detections –Glare – false detections, object shape and trajectory distortions –Sudden illumination changes (cloud movements) – false detections –Low contrast or color saturation

Moving Object Detection Approaches Optical Flow Analysis: Characteristics of flow (velocity) vectors of moving objects over time are used to detect changed regions. Advantage: can be used in the presence of camera motion. Disadvantage: usually computationally expensive & aperture problem. Change Detection Background subtraction: Moving regions are detected through difference between the current frame and a reference background image. | frame i -Background i |>Th Advantage: provides the most complete feature data. Disadvantage: sensitive to dynamic scene changes due to lighting and extraneous events and cannot handle global motion. Temporal differencing: Similar to background subtraction but the estimated background is the previous frame. | frame i -frame i-1 |>Th Advantage: very adaptive to dynamic environments. Disadvantage: has problems in extraction of all relevant feature pixels (aperture problem).