Presentation on theme: "MIT AI Lab Paul Viola NTT Visit: Image Database Retrieval Variable Viewpoint Reality Professor Paul Viola Collaborators: Professor Eric Grimson, Jeremy."— Presentation transcript:
MIT AI Lab Paul Viola NTT Visit: Image Database Retrieval Variable Viewpoint Reality Professor Paul Viola Collaborators: Professor Eric Grimson, Jeremy De Bonet, John Winn, Owen Ozier, Chris Stauffer, John Fisher, Kinh Tieu, Dan Snow, Tom Rikert, Lily Lee, Raquel Romano, Janey Hshieh, Mike Ross, Nick Matsakis, Jeff Norris, Todd Atkins Mark Pipes
MIT AI Lab Paul Viola Overview of Visit Morning: Image Database Retrieval –Gatekeeper: Face detection and recognition –Complex Feature Image Database Retrieval (Tieu) –Flexible Template Retrieval (Yu) Interlude –Video/Audio Source Separation (Fisher) –Mathematical Expression Recognition (Matsakis) Lunch Visit Prof. Brooks lab
MIT AI Lab Paul Viola Overview of Visit - 2 Afternoon: Variable Viewpoint Reality –Real-time 3D reconstruction of people (Snow) –Automatic camera calibration (Snow + Lee) –Tracking of articulate human models (Lee + Winn) –Modeling of human dynamics (Viola + Fisher)
MIT AI Lab Paul Viola Gatekeeper: Receptionist & Security Greet guests Direct people to their destinations Recognize employees Turn back unauthorized visitors
MIT AI Lab Paul Viola Gatekeeper in action … Gatekeeper Movie
MIT AI Lab Paul Viola Gatekeeper is a constant observer… Professor Paul Viola
MIT AI Lab Paul Viola Detecting faces is very difficult
MIT AI Lab Paul Viola Detecting and Recognizing Faces Key Difficulty: Variation in Pose State of the art: generalized templates –Neural Networks / Deformable Templates / etc. Templates have difficulty with pose variation… –Rotation, scale, complex deformation Must reduce the dependence on relative pose. Approach: Detecting people as a statistical distribution of multi-scale features
MIT AI Lab Paul Viola Statistical Distribution of Multi-scale Features Wavelet Pyramid The distribution of multi-scale features determines appearance
MIT AI Lab Paul Viola Multi-scale Wavelet Features A multi-scale feature associates many values with each pixel in the image
MIT AI Lab Paul Viola I MODEL I TEST Cross Entropy Discrimination via Cross Entropy
MIT AI Lab Paul Viola BTR70-C71T72-132 Motivation: Finding vehicles in clutter Supported by Darpa: IU/ATR initially MSTAR Extension
MIT AI Lab Paul Viola Can also be used for segmentation…
MIT AI Lab Paul Viola Synthesis Results Original Texture The multi-scale statistical model can be used to generate new example textures
MIT AI Lab Paul Viola Synthesis Procedure Step 1: Build analysis pyramid Input Image Note: We are using only the Gaussian pyramid here! Normally we use an oriented pyramid... 64x64 2x2
MIT AI Lab Paul Viola Synthesis Procedure Step 2: Build synthesis pyramid
MIT AI Lab Paul Viola Synthesis Procedure Step 2a: Fill in the top... Pixels are generated by sampling from the analysis pyramid.
MIT AI Lab Paul Viola Synthesis Procedure Step 2b: Fill in subsequent levels Pixels are generated by conditional sampling (dependent on the parent).
MIT AI Lab Paul Viola Synthesis Procedure Finish the pyramid Decisions made at low resolutions generate discrete features in the final image.
MIT AI Lab Paul Viola Detection Results Non-face test images Web face test images
MIT AI Lab Paul Viola Pruning the density estimator Reduce the number of bins through clustering 1000 bins or less! Result: Detection/Classification is faster than template correlation
MIT AI Lab Paul Viola Key facial features - determined automatically - located automatically Multi-scale features which are come from the face model can be automatically detected for many individuals
MIT AI Lab Paul Viola New Face Recognition Algorithm Measure the occurrence and location of “key” facial features. Facial identity depends both on the types of features and their location. Relation to Active Search… –Match measure is a histogram of multiscale features –Like color histogram, Active Search can be used...
MIT AI Lab Paul Viola Presentation on Image Database Technology