Presentation on theme: "A Unified Framework for Context Assisted Face Clustering"— Presentation transcript:
1 A Unified Framework for Context Assisted Face Clustering Liyan Zhang, Dmitri V. Kalashnikov, Sharad MehrotraDepartment of Computer ScienceUniversity of California, Irvine
2 Introduction User Feedback Face Clustering Face Tagging Human is CenterExplosion of Media DataUser FeedbackFace ClusteringFace TaggingThe prevalence of digital cameras as well as the emergence of online mediaweb sites such as Flickr, Picasa, Fackbook, Twitter, etc., makes the creation, storageand sharing of multimedia content much easier than before, which leadsto the explosion of massive media data. As the continue growing of the size ofpersonal media collections, the problem of media organization, management andretrieval has become a much more pressing issue. Among most photo collections,human is usually the focus of images. To better understand and manage these
3 Outline Introduction to Face Clustering Traditional Approaches for Face ClusteringThe Proposed Context Assisted FrameworkExperimental ResultsConclusions and Future Work
4 Face Appearance based Approach Facial FeaturesDetected Faces……Clustering ResultsClustering AlgorithmFace Similarity Graph
5 Appearance based Face Clustering Results Good Clustering ResultsHigh Precision,High RecallTight Clustering ThresholdHigh Precision, Low Recallloose Clustering ThresholdLow Precision, High RecallToo Much Merging Work!
6 Drawbacks of Facial Similarities Same People LookDifferentDifferent PoseDifferent ExpressionDifferent IlluminationDifferent OcclusionDifferent People LookThe SameBoyGirl
7 Context Information Helps Common Scene:Geo LocationCaptured TimeImage BackgroundSocial Context:People Co-occurHuman Attributes:AgeEthnicityGenderHair…Clothing:Cloth color
8 Related Work People Co-occurrence  Human Attributes  Clothing ContextPrior workSingleContextHeterogeneousFace LevelCluster LevelSingle Context Type Y. J. Lee and K. Grauman. Face discovery with social context. In BMVC, 2011. N. Kumar and et al. Describable visual attributes for face verification and image search. In IEEE TPAMI, 2011. A. Gallagher and T. Chen. Clothing cosegmentation for recognizing people. In IEEE CVPR, 2008.Heterogeneous Context Feature
9 The Framework …… …… …… … cont Initial Clusters : High Precision, Low RecallPhoto CollectionDetected FacescontCommon ScenePeople Co-occurrenceHuman AttributesClothing………Iterative Merging……Final Clusters: High Precision, High Recall
13 People Co-occurrence I1 I2 f 5 f 2 f 1 f 4 f 3 f 8 f 7 f 6 1 1 1 1 1 1 Cluster Co-Occurrence Graph111111
14 Human Attributes same diff 73-D 73-D N. Kumar and et al. Describable visual attributes for face verification and image search. In IEEE TPAMI, 2011.
15 Learn Weights From Dataset! Human AttributesAttributeC5Only One ChildMany Children!AGE attributeC5cosineSimilar?f 1f 2f 3AttributeDifferent AttributesDifferent WeightsBootstrapping:Learn Weights From Dataset!
16 Human Attributes diff Face Attributes Label C1 ~ C1 Train Classifier
17 Clothing Time Sensitive! Diff Same Similarity from clothes Cloth color hist similarityTime diffTime slot thresholdTime diff << STime diff >> S
19 Cluster-Level Context Constraints Context FeaturesTime & LocationTime: t s IndoorTime: (t+1)s OutdoorDiffDiffDistinct AttributesCluster-Level Context ConstraintsDiffCo-occurred people
20 Single Context Feature Fails People Co-occurrenceCommon SceneSameSameDiffDiffCo-occurred PeopleDifferent AttributesDifferent Clothing
21 Integration is Required Context SimilaritiesContext ConstraintsIntegrate ?Aggregation?Set a rule?Context FeaturesY=a b c d e …The importance of features differ with different dataset!MergeNotLearn Rules from Each Dataset!
22 How to Learn Rules? …… … Manually Label Learning Rules Apply Rules Training DatasetContext ConstraintsDiff PairsApply rulesLearningRulesTraining DatasetAutomatic LabelSplit InitialclustersSame PairsBootstrapping……Initial Clusters : High Precision, Low RecallFacial FeaturesPhoto CollectionLearn from dataItself!contCommon ScenePeople Co-occurrenceHuman AttributesClothing…Context Similarity& Constraint
23 Example of Automatic Labeling SplittingTrainingpairsLabelSameDiffDiffsamediffCost-sensitiveDTC
24 1st Splitting—Training--Predicting pairsLabelSameDiffDiffPredictingpairspredictSameDiff…( ):5 sameCost-sensitiveDTC( ):1 same
25 2nd Splitting—Training--Predicting pairsLabelSameDiffDiffPredictingpairspredictSameDiff…( ):4 sameCost-sensitiveDTC( ):0 same
26 Combine results pairs predict … 1st Time C1-C3: 5 same C2-C3: 1 same Diff…1st TimeC1-C3:5 sameC2-C3:1 sameC1-C3:9 samepairspredictSameDiff…MergeC1-C3C2-C3:1 same2nd TimeC1-C3:4 sameC2-C3:0 same
27 Unified Framework … Faces Photo Album splitting training prediction ExtractedFacesContextFeaturesIterative MergingsplittingtrainingpredictionFacialPureclusterssplittingtrainingpredictionFinalDecision…splittingtrainingpredictionYESMergePairs?NoResults