I think this is a giraffe. What do you think? No, its neck is too short for it to be a giraffe. Ah! These must not be giraffes either then. [Animals with even shorter necks] …… Current belief Focused feedback Knowledge of the world Feedback on one, transferred to many Learner learns better from its mistakes Accelerated discriminative learning with few examples Learner learns better from its mistakes Accelerated discriminative learning with few examples
Communication Need a language that is Machine understandable Human understandable Attributes! Mid-level shareable Visual Semantic
Proposed Active Learning Concepts to teach: C1C1 C2C2 CKCK … Unlabeled pool of images Classifiers: h1h1 h2h2 hKhK … [Label-feedback] [Attributes-based feedback] Attribute predictors: a1a1 a2a2 aMaM … [Predicted Label] Any feature space Any discriminative learning algorithm
Relative Attributes [Parikh and Grauman, ICCV 2011] Openness Unlabeled pool of images Attribute predictors: a1a1 a2a2 aMaM … Image features Parameters
Attributes-based Feedback Unlabeled pool of images No, It is too open to be a forest Attribute predictors: a1a1 a2a2 aMaM … Forest Openness Not Forest
Attributes-based Feedback Unlabeled pool of images No, It is too open to be a forest Attribute predictors: a1a1 a2a2 aMaM … Forest Classifiers: h1h1 h2h2 hKhK … Not Forest
Proposed Active Learning Concepts to teach: C1C1 C2C2 CKCK … Unlabeled pool of images Classifiers: h1h1 h2h2 hKhK … [Label-feedback] [Attributes-based feedback] Attribute predictors: a1a1 a2a2 aMaM … [Predicted Label]
Label-based Feedback Not our contribution Experiment with different scenarios Benefits of attributes-based feedback – Small when label-based is very informative – Large when label-based feedback is weak
Unlabeled pool of images Forest Label-based Feedback Accept: Yes, this is a forest. – Strong: It is not anything else Example: Classification – Weak: It can be other things Example: Annotation
Label-based Feedback Reject: – Strong: No, it is a coast. Example: Classification with few classes – Weak: No, this is not a forest. Example: Large-scale classification Example: Biased binary classification 4 different scenarios in experiments Unlabeled pool of images Forest
Datasets Datasets and relative attribute predictors from [Parikh and Grauman, ICCV 2011]
Settings Feedback from MTurk Features: – Raw image features (gist, color) – Attribute scores Category classifiers: SVM with RBF kernel Results on – 2 datasets x 2 features x 4 label-feedback scenarios – Show 2 here, rest in paper. This image doesnt have enough perspective to be a street scene.
Results Scenes, Image features, Weak label-feedback # iterations Accuracy 1/4 th ! More results in the paper.
Conclusion Attributes for providing classifier feedback Novel learning paradigm with enhanced human-machine communication Discriminative learning + domain knowledge Learning with few examples Connections to semi-supervised learning – Shrivastava, Singh and Gupta: Up Next!
Negative Feedback Many reasons come together to make a concept – Hard to describe why an image is a concept One reason can break a concept – Easier to describe why an image is not a concept (Arguably) waste to give feedback when right
Discriminative – No domain knowledge is conveyed – Many training images – Discriminative model – Classification in any feature space – State-of-art performance Related Work Zero-shot learning – Convey domain knowledge – Zero training images – Generative model – Classification in attribute space – Performance compromised Proposed paradigm – Convey domain knowledge to transfer – Few training images – Discriminative model – Classification in any feature space – Performance maintained bea r turtlerabbit furry big [Lampert et al., CVPR 2009] C Smiling Age S J H C is younger than H C smiles more than H M M is younger than J M smiles more than J [Parikh and Grauman, ICCV 2011] + + + – – –
Related Work Focused discrimination – Mining of hard negatives [Felzenszwalb 2010] – To understand classifier [Golland 2001] – Here, supervisor provides the discriminative direction by verbalizing semantic knowledge [Golland, NIPS 2001][Felzenszwalb et al., CVPR 2008]
Related Work What we are not doing: Collecting deeper annotations of images Segmentation masks [Russell 2008], Parts [Farhadi 2010], Pose [Bourdev 2009], Attributes [Kumar 2009] We use attributes for broad propagation of category labels to unlabeled images
Related Work What we are not doing: Actively interleaving attribute annotations Object & attributes [Kovashka 2011], Image, boxes & segments [Vijaynarsimhan 2008], Parts & attributes [Wah 2011], etc. Human-in-the-loop at test time [Branson 2010] Our supervisor provides additional information at training time which is leveraged for better category models
Related Work Rationales – Human feature selection in NLP [Raghavan 2005] – Spatial and attribute rationales [Donahue 2011] – Restricted to classification in attribute space – We can operate in any feature space [Donahue and Grauman, ICCV 2011]
Imperfect Attribute Predictors In the end, all images labeled with ground truth Discriminative training can deal with outliers Attributes are pre-trained, and so unlikely to be severely flawed Experiments: used predictors directly from Parikh and Grauman, 2011
Large-scale Classification Categories may require expert knowledge, attributes need not Can show a few exemplars to verify category
Results Faces, Classification, Attribute features (overestimate) # iterations Accuracy
Label-based Feedback ClassificationLarge-scale Classification Annotation Street Outdoor City Grayscale …. Biased Binary Classification Simulate the different scenarios in responses Benefits of attributes-based feedback Small when label-based is very informative Large when label-based feedback is weak Simulate the different scenarios in responses Benefits of attributes-based feedback Small when label-based is very informative Large when label-based feedback is weak
Results Scenes, Annotation, Image features # iterations Accuracy
Results Faces, Biased binary classification, Attribute features # iterations Accuracy More results in the paper.
Traditional Active Learning Concepts to teach: C1C1 C2C2 CKCK … Unlabeled pool of images Classifiers: h1h1 h2h2 hKhK … [Label] Highest Entropy
Your consent to our cookies if you continue to use this website.