1. Finding Celebrities in Billions of Web Images 云飞 2012-12-13

2. 一、 label an input image with a list of celebrities. 二、 the celebrity names are assigned to the faces by label propagation on a facial similarity graph. Overview

3. 本文的优点： 1 、 the proposed image annotation system is capable of labeling names to general web images. 2 、 our name assignment algorithm does not impose any assumption on the facial feature distribution. 3 、 not only visual cues are used. Overview

4. 1. determine, by identifying celebrity names from surrounding text. 2. given a set of names, assign the names to the faces in the input image. Overview

5. A. Image Annotation System 1) construct a vocabulary; 2) discover all webpages hosting its near-duplicates; 3) use the vocabulary to filter the surrounding text. – Advances ： 1)effective; 2)remove noise. – Annotated images: 1)SFSN 2)SFMN 3)MF Overview

6. B. Multimodal Name Assignment The context likelihood incorporates the information from surrounding text by using the confidence scores estimated by the image annotation system. Overview

7. IMAGE ANNOTATION SYSTEM Goal: label an input image with a list of celebrities who may appear in the image. A. Constructing a Large-Scale Celebrity Name Vocabulary B. Discover Related Webpages by Near-Duplicate Image Retrieval C. Annotating Images by Mining Surrounding Text of Related Webpages

8. IMAGE ANNOTATION SYSTEM A.Constructing a Large-Scale Celebrity Name Vocabulary 1)Wikipedia 首段 信息框 标签 2)Entitycube

9. IMAGE ANNOTATION SYSTEM B. Discover Related Webpages by Near- Duplicate Image Retrieval – divide and conquer strategy 图片分成 n×n 降维 阈值化

10. IMAGE ANNOTATION SYSTEM C. Annotating Images by Mining Surrounding Text of Related Webpages 1) Type of names ； 2) Type of surrounding text ； 3) Frequency versus ratio ；

11. A. Notation B. Overview of the Assignment Model C. Label Propagation from SFSN Images p(Y|F) D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) E. Normalization by Name Prior p(Y) F. Implementation Detail: Face Representation MULTIMODAL NAME ASSIGNMENT

12. A. Notation – faces in image I n – denote the face labels as

13. B. Overview of the Assignment Model – the confidence for label

14. C. Label Propagation from SFSN Images p(Y|F) – how to propagate labels from SFSN images to SFMN and MF images

15. D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) 1) For each image-level name v k, generate a binary variable z k from p(v k |T) as defined in (3) to indicate whether v k appears in image I. 2) If z k =1, generate m k faces of name v k in image I from p(m|z; λ) as defined in (13).

16. E. Normalization by Name Prior p(Y) – p(Y) represents the prior of names.

17. F. Implementation Detail: Face Representation the appearance of each face is described by local binary pattern (LBP). the face image is divided into small regions from which the LBP features are extracted and concatenated into a single feature histogram. pply PCA to reduce the dimension of face descriptor from over 3000 to 500 dimensions.

18. Evaluation