An MPEG-7 Based Semantic Album for Home Entertainment Presented by Chen-hsiu Huang 2003/08/12 Presented by Chen-hsiu Huang 2003/08/12

Introduction  The digital album should not only process the meaningless 0/1 bits but also realize the semantic information from media.  It could be much better if computer knows which region is more important, which photo has any important person in it, or which photos have close relation with current browsing one.  The face detection & recognition technology have developed for years, can we query our daily photos by face?

Core Functionalities  Query image by face –Face detection & recognition  Photo Focus identification –Smart Thumbnail  Photo Similarity –Relative photos  Photo Grammar –Not yet done

Query Images by Face  Steps to achieve query by face: –Find out the faces in photos –Build the face databases –Training face databases –Recognize faces in photos –Query images by Face PS: We use Intel OpenCV as face detection & recognition module

Photo Focus  For photos with people, human faces are surely our focus when viewing.  The user attention model has applied to find some saliency points: –Red: Intensity based –Green: Color based –Blue: Skin color based –Texture based

Smart Thumbnail  Direct Scaling –Traditional way of creating thumbnail  Focus based –Cropping the focus region first, then scaling –Better then direct scaling, but not so good  Adaptive selection –For each face & saliency points, a weighting function was applied to calculate its importance. –User can select the cropping ratio, the cropping region is adaptive decided according to the weighting value.

Focus Based Adaptive Selection Direct Scale

Photo Similarity  Color Layout Descriptor –It is designed to efficiently represent spatial distribution of colors  Dominant Color Descriptor –The representative colors in an image or image region  By using the human faces information and MPEG-7 descriptors, we can calculate the similarities between images.

System Diagram Face detection & reorganization User attention model Saliency Map MPEG-7 Visual Descriptors Query by Face Photo Focus & Smart Thumbnail Photo Similarity Photo Grammar Evaluation We can get more semantic meanings from low level features by combining those kernel modules.

In the Future  The album should be able to cope with different type photos.  The album system can be improved both systematic side and component side.  The album should be able to process other media type such as audio and video.  The album syntax should be fully conform to the MPEG-7 standard.

The End  Any recommendation is welcomed.  Thank you.

Adaptive Selection  For all the visual objects (faces, saliency points), calculate its importance by:  When adaptive selection, sort those visual objects by importance, dropping the least import object to achieve the goal cropping ratio.