Background Removal David Harwin Adviser: Petros Faloutsos

The Problem Many applications use video to record data However, lots of data is wasted on representing the background, which can be unnecessary at best and distracting at worst Removing background can improve clarity and reduce file size

The Solution Capture an image of the background with no foreground images Process each subsequent frame of video by comparing it to the basis image Portions of the image determined to be in the background are replaced with arbitrary data Use a scene graph to include higher-level processing to improve accuracy

Applications and Goals Medical recordings: removing extraneous distractions  quality of foreground data is critical  not constrained to real-time processing Visual effects processing: simpler or alternative to chroma key  quality is important  allows long processing time Videoconferencing: improve performance of video compression techniques (interframe and intraframe)‏  some artifacts are acceptable  requires real-time or near-real-time processing For this project, the goal is accuracy and preservation of foreground data

Difficulties Must allow some motion in the background (caused by wind, etc.)‏ Must allow slight lighting changes in the background Must keep tolerances high enough to preserve foreground

Tasks Involved Set up environment for handling and processing video (cross-platform is a plus)‏ Implement a background removal algorithm Compare with existing implementations Extend to stereo, adding 3D geometry, converting 2D to 3D, etc.

Deliverables Project length: 2 quarters Checkpoint 1:  Have framework for video input and processing Checkpoint 2:  Have background removal implementation  Have comparison with existing implementations