Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Liyan Zhang et al. Liyan Zhang, Dmitri V. Kalashnikov, Sharad Mehrotra Department of Computer Science University of California, Irvine A Unified Framework.

Similar presentations


Presentation on theme: "1 Liyan Zhang et al. Liyan Zhang, Dmitri V. Kalashnikov, Sharad Mehrotra Department of Computer Science University of California, Irvine A Unified Framework."— Presentation transcript:

1 1 Liyan Zhang et al. Liyan Zhang, Dmitri V. Kalashnikov, Sharad Mehrotra Department of Computer Science University of California, Irvine A Unified Framework for Context Assisted Face Clustering

2 2 Liyan Zhang et al. Introduction Explosion of Media Data Human is Center Face Clustering Face Tagging User Feedback

3 3 Liyan Zhang et al. Outline Introduction to Face Clustering Traditional Approaches for Face Clustering The Proposed Context Assisted Framework Experimental Results Conclusions and Future Work

4 4 Liyan Zhang et al. Face Appearance based Approach Facial Features Face Similarity Graph Clustering Algorithm Detected Faces … Clustering Results

5 5 Liyan Zhang et al. Appearance based Face Clustering Results Good Clustering Results High Precision,High Recall Tight Clustering Threshold High Precision, Low Recall loose Clustering Threshold Low Precision, High Recall Too Much Merging Work!

6 6 Liyan Zhang et al. Drawbacks of Facial Similarities Same People Look Different Different PoseDifferent Expression Different IlluminationDifferent Occlusion Different People Look The Same BoyGirlBoyGirl

7 7 Liyan Zhang et al. Context Information Helps Common Scene: Geo Location Captured Time Image Background Social Context: People Co-occur Human Attributes: Age Ethnicity Gender Hair … Clothing: Cloth color

8 8 Liyan Zhang et al. Related Work [1] Y. J. Lee and K. Grauman. Face discovery with social context. In BMVC, [3] N. Kumar and et al. Describable visual attributes for face verification and image search. In IEEE TPAMI, [2] A. Gallagher and T. Chen. Clothing cosegmentation for recognizing people. In IEEE CVPR, People Co-occurrence [1] Clothing [2] Human Attributes [3] Heterogeneous Context FeatureSingle Context Type Face LevelCluster Level Context Prior work Context Heterogeneous Single

9 9 Liyan Zhang et al. The Framework Photo Collection Detected Faces … Initial Clusters : High Precision, Low Recall … Iterative Merging cont Common Scene People Co-occurrence Human Attributes Clothing … … Final Clusters: High Precision, High Recall

10 10 Liyan Zhang et al. Context Features Extraction Cluster level Common ScenePeople Co-occurrenceHuman AttributesClothing Context Similarities Context Constraints Integrate Same? Diff ?

11 11 Liyan Zhang et al. Common Scene Image captured time, camera model, image visual features Common Scene Same people I1I1 I2I2 I3I3 I1I1 I2I2 C1C1 C2C2

12 12 Liyan Zhang et al. People Co-occurrence same diff

13 13 Liyan Zhang et al. People Co-occurrence Cluster Co-Occurrence Graph I2I2 I1I f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8

14 14 Liyan Zhang et al. Human Attributes samediff 73-D N. Kumar and et al. Describable visual attributes for face verification and image search. In IEEE TPAMI, 2011.

15 15 Liyan Zhang et al. Human Attributes Attri bute C5 f 1 f 2 f 3 Attri bute Similar? C5 cosine Only One Child Many Children! AGE attribute Bootstrapping: Learn Weights From Dataset! Different Attributes Different Weights

16 16 Liyan Zhang et al. Human Attributes FaceAttributesLabel C1C1 C1C1 C1C1 ~ C 1 Attri bute C5 f 1 f 2 f 3 Attri bute f 4 f 5 f 6 f 7 f 8 diff Attri bute Train Classifier C1 ?

17 17 Liyan Zhang et al. Clothing Similarity from clothes Cloth color hist similarity Time diff Time slot threshold Time Sensitive! DiffSame Time diff << S Time diff >> S

18 18 Liyan Zhang et al. Context Features Cluster-Level Context Similarities Common ScenePeople Co-occurrence Human Attributes Clothing

19 19 Liyan Zhang et al. Context Features Cluster-Level Context Constraints Diff Co-occurred people Diff Distinct Attributes Time & Location Time: t s Indoor Time: (t+1)s Outdoor Diff

20 20 Liyan Zhang et al. Single Context Feature Fails Common Scene People Co-occurrence Diff Same Different Attributes Different Clothing Co-occurred People Diff Same

21 21 Liyan Zhang et al. Integration is Required Aggregation? Set a rule? Context Similarities Context Constraints Integrate ? Context Features Merge Not Y=a +b +c +d +e +… The importance of features differ with different dataset! Learn Rules from Each Dataset!

22 22 Liyan Zhang et al. How to Learn Rules? Photo Collection Split Initial clusters Same Pairs Manually Label Learning Rules Apply Rules Training Dataset … Initial Clusters : High Precision, Low Recall Facial Features cont Common Scene People Co-occurrence Human Attributes Clothing … Context Similarity & Constraint Learn from data Itself! Apply rules Learning Rules Training Dataset Automatic Label Context Constraints Diff Pairs Bootstrapping

23 23 Liyan Zhang et al. pairsLabel Same Diff Cost-sensitive DTC Splitting Training Diff Example of Automatic Labeling same diff same diff same

24 24 Liyan Zhang et al. pairsLabel Same Diff Cost-sensitive DTC pairspredict Same Diff Same Diff …… ( ): 5 same ( ): 1 same Splitting Training Predicting Diff 1 st Splitting—Training--Predicting

25 25 Liyan Zhang et al. pairsLabel Same Diff Cost-sensitive DTC pairspredict Same Diff …… ( ): 4 same ( ): 0 same Splitting Training Predicting Diff 2 nd Splitting—Training--Predicting

26 26 Liyan Zhang et al. Combine results C1-C3: 5 same C2-C3: 1 same C1-C3: 4 same C2-C3: 0 same C1-C3: 9 same C2-C3: 1 same Merge C1-C3 pairspredict Same Diff Same Diff …… pairspredict Same Diff …… 1 st Time 2 nd Time

27 27 Liyan Zhang et al. Unified Framework Pure clusters splittingtrainingprediction splittingtrainingprediction splittingtrainingprediction … Final Decision Extracted Faces Merge Pairs? YES No Results Iterative Merging Photo Album Facial Context Features

28 28 Liyan Zhang et al. Experiment Datasets Gallagher Wedding Surveillance

29 29 Liyan Zhang et al. Evaluation Metrics B-cubed Precision and Recall

30 30 Liyan Zhang et al. Performance Comparison Facial Features Photo Album Context Features Pure Clusters Splitting Training Predicting Process Merge Decision Update Our Approach: Precision Recall Picasa: Cluster Threshold 50  95 Different Clusters Affinity Propagation: Context Similarities Aggregation Facial Similarities Different Parameter: p Different Clusters

31 31 Liyan Zhang et al. Results

32 32 Liyan Zhang et al. Results High Precision Higher Recall

33 33 Liyan Zhang et al. Results High Precision, 662 clusters 31 Real Person, 631 Merging High Precision, 203 clusters 31 Real Person, 172 Merging 4 Times

34 34 Liyan Zhang et al. Results Less Clusters Less Manual Merging

35 35 Liyan Zhang et al. Results

36 36 Liyan Zhang et al. Conclusion and Future Work Heterogeneous Context Features Context Constraint Co-occur People Distinct Attributes Time & Space Context Similarity Common Scene People Co-occur Human Attributes Clothing Single Context Feature Context Similarity Prior work Our Approach Efficiency?User Feedback? Break points for precision dropping? Future work Bootstrapping Integration Iterative Merging High precision High recall

37 37 Liyan Zhang et al. Thank you! Questions?


Download ppt "1 Liyan Zhang et al. Liyan Zhang, Dmitri V. Kalashnikov, Sharad Mehrotra Department of Computer Science University of California, Irvine A Unified Framework."

Similar presentations


Ads by Google