1. Elliptical Head Tracking Using Intensity Gradients and Color Histograms Stan Birchfield Stanford University Autodesk Advanced Products Group http://vision.stanford.edu/~birch

2. PROBLEM TILT PAN ZOOM CHALLENGES: * rotation * multiple people * zoom APPLICATIONS: * video conferencing * distance learning

3. PREVIOUS METHODS FLESH- COLORED OBJECTS MULTIPLE MOVING PEOPLE ARBITRARY CAMERA MOVEMENT OUT-OF-PLANE ROTATION 1. TEMPLATE [Hager & Belhumeur, 1996] YYYN 2. FLESH COLOR [Fieguth & Terzopoulos, 1997] NNYN 3. BACKGROUND DIFFERENCING [Graf et al., 1996] YNNY Method Criterion

4. CUES: COLOR MOTION TEXTURE INTERIORBOUNDARY COMPLEMENTARY CRITERIA INTENSITY EDGES DEPTH & MOTION. DISCONTINUITIES

5. APPLICATION: 1. Interesting, useful 2. Well-connected to other body parts WHY FOCUS ON THE HEAD? GEOMETRIC: 1. Nearly rigid 2. Nearly ellipsoid Easy to model

6. HEAD MODEL (x,y) Ellipse: vertical aspect ratio = 1.2  state s = (x,y,  ) SEARCH velocity prediction  LOCAL HEAD SEARCH GRADIENTCOLOR SEARCH RANGE

7. TWO CHOICES: 1. MAGNITUDE 2. DOT PRODUCT NORMALIZATION GRADIENT MODULE ellipse normalgradient

8. COLOR MODULE COLOR SPACE HISTOGRAM INTERSECTION [Swain & Ballard 1991] NORMALIZATION B-G (8 bins)G-R (8 bins) B+G+R (4 bins) MODEL CURRENT INTERSECTION SKINHAIR

9. SUMMARY OF ALGORITHM OFF-LINE: 1. Manually place head within ellipse 2. Store model histogram RUN TIME: 1. At each hypothesized location, compute - Sum of gradient around perimeter - Histogram intersection 2. Move ellipse to location that maximizes sum of two criteria

10. COMPARISON OF MODULES Controls pan, tilt, zoom Handles textured backgrounds More robust Large basin of attraction Controls pan, tilt Keeps off neck Scale in front of flesh-colored object Scale when back turned COLORGRADIENT

11. BASIN OF ATTRACTION Gradient confused, pulls to leftColor pulls to right

12. COMPUTING TIME Real time (30 Hz) Computing time per frame (ms) Search range (on a 200 MHz Pentium Pro)

13. CONCLUSION SUCCESSES: 1. Tracks head in real time on standard hardware 2. Insensitive to - full 360-degree out-of-plane rotation - arbitrary camera movement (including zoom) - multiple moving people - severe but brief occlusion - hair/skin color, hair length, facial hair, glasses FUTURE WORK: 1. Speed (computer speed and NTSC video standard) 2. Color adaptation, but imprecise localization 3. No explicit model of occlusion