 Introduction  Principles of context – aware saliency  Detection of context – aware saliency  Result  Application  Conclusion

 How to describe a figure / picture ?  Description: What most people think is Important or Salient.

 First glance: human attention. EX: auto focusing.  Dominant object EX: object recognition/segmentation.  context of the dominant objects: image classification, summarization of a photo collection, thumb nailing, and retargeting.

 Salient regions are distinctive with respect to both their local and global surroundings.  Prioritize regions close to the foci of attention. -> Maintains the background texture. (Gestalt Law)

 Retargeting  Summarization

 Principles for context-aware saliency  Algorithm  Applicability

 1. Local low-level considerations, including factors such as contrast and color.  2. Global considerations, which suppress frequently occurring features, while maintaining features that deviate from the norm.

 3. Visual organization rules, which state that visual forms may possess one or several centers of gravity about which the form is organized.  4. High-level factors, such as human faces.

 D.Walther and C. Koch. Modeling attention to salient protoobjects.[2006]  X. Hou and L. Zhang. Saliency detection: A spectral residual approach.[2007]  T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum. Learning to Detect A Salient Object.[2007]

 Areas that have distinctive colors or patterns should obtain high saliency.--P1  frequently-occurring features should be suppressed.--P2  The salient pixels should be grouped together, and not spread all over the image.--P3

 Consider a single patch of scale r at each pixel. Thus, a pixel i is considered salient if the appearance of the patch pi centered at pixel i is distinctive with respect to all other image patches.

 Background pixels (patches) are likely to have similar patches at multiple scales.  Incorporating multiple scales to further decrease the saliency of background pixels, improving the contrast between salient and non-salient regions.

 Gestalt laws: visual forms may possess one or several centers of gravity about which the form is organized.

 Enhancement factors: 1.Recognized objects 2.Face detection

 3 cases:  Images show a single salient object over an uninteresting background.  Images where the immediate surroundings of the salient object shed light on the story the image tells.  Images of complex scenes.  Compare method:  D.Walther and C. Koch. Modeling attention to salient protoobjects.[2006]  X. Hou and L. Zhang. Saliency detection: A spectral residual Approach.[2007]

 Image retargeting  Summarization through collage creation

 Resizing an image by expanding or shrinking the non-informative regions.  Seam carving M. Rubinstein, A. Shamir, and S. Avidan. Improved seam carving for video retargeting.[2008]  Context-aware saliency

 The salient objects as well as informative pieces of the background should be maintained in summaries.

S. Goferman, A. Tal, and L. Zelnik-Manor. Puzzle-like collage.[2010]  3 stages:  Compute the saliency maps for images.  Extracts regions-of-interest by considering both saliency and image edge information.  Assemble non-rectangular ROIs.

 Propose a new type of saliency: context-aware saliency  Evaluate in 2 applications: retargeting 、 summarization

Thanks for attention!