Download presentation

Presentation is loading. Please wait.

Published byMiles Emm Modified over 2 years ago

1
Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance Dhruv Batra, Carnegie Mellon University Adarsh Kowdle, Cornell University Devi Parikh, Toyota Technological Institute Jiebo Luo, Eastman Kodak Company Tsuhan Chen, Cornell University

2
Outline 1 Introduction 2 iCoseg: Energy Minimization 3 iCoseg: Guiding User Scribbles 4 The CMU-Cornell iCoseg Dataset 5 Experiments 6 Interactive Co-segmentation for Object-of- Interest 3D Modeling

3
Introduction We develop an algorithm that allows users to decide what foreground is, and then guide the output of the co-segmentation algorithm towards it via scribbles

4
Outline 1 Introduction 2 iCoseg: Energy Minimization 3 iCoseg: Guiding User Scribbles 4 The CMU-Cornell iCoseg Dataset 5 Experiments 6 Interactive Co-segmentation for Object-of- Interest 3D Modeling

5
Energy Minimization 1. Data (Unary) Term : indicating the cost of assigning a superpixel to foreground and background classes 2. Smoothness (Pairwise) Term : used for penalizing label disagreement between neighbours I (·) : an indicator function that is 1(0) if the input argument is true (false) d ij : the distance between features at superpixels i and j β : a scale parameter

6
Outline 1 Introduction 2 iCoseg: Energy Minimization 3 iCoseg: Guiding User Scribbles 3.1 Uncertainty-Based Cues 3.2 Scribble-Based Cues 3.3 Image-Level Cues 3.4 Combined Recommendation Map 4 The CMU-Cornell iCoseg Dataset 5 Experiments 6 Interactive Co-segmentation for Object-of- Interest 3D Modeling

7
Image cues: Segment Size Codeword Distribution Recommendation Map

8
3.1 Uncertainty-Based Cues 1. Node Uncertainty (NU):Fitting A1 = {GMM f,GMM b } to the labelled superpixel features. Using this learnt A1, for each superpixel we normalize the foreground and background likelihoods to get a 2-class distribution and then compute the entropy of this distribution. 2. Edge Uncertainty (EU):To feed unlabelled data- points to a set of classifiers and request label for the datapoint with maximal disagreement among classifier outcomes.

9
3.1 Uncertainty-Based Cues 3. Graph-Cut Uncertainty(GC): Capture the confidence in the energy minimizing state returned by graph cuts. 3.2 Scribble-Based Cues 4. Distance Transform over Scribbles (DT): Compute the distance of every pixel to the nearest scribble location.

10
3.2 Scribble-Based Cues 5. Intervening Contours over Scribbles (IC): The value of this cue at each pixel is the maximum edge magnitude in the straight line to the closest scribble.

11
3.3 Image-Level Cues 6. Segment Size (SS): When very few scribbles are marked, energy minimization methods typically overs- mooth and results in “whitewash” segmentations (entire image labelled as foreground or background). 7. Codeword Distribution over Images (CD):Motivation being that scribbling on images containing more diversity among features would lead to better foreground /background models. To compute this cue, we cluster the features computed from all superpixels in the group to form a codebook.

12
3.4 Combined Recommendation Map Learning a mapping F :φ i → ϵ i, φ i is the 7-dimensional feature vector for superpixel i ϵ i is the error indicator vector which is 1 if the predicted segmentation at node ϵ i is incorrect, and 0 otherwise. We chose logistic regression as the form of this mapping.

13
Outline 1 Introduction 2 iCoseg: Energy Minimization 3 iCoseg: Guiding User Scribbles 4 The CMU-Cornell iCoseg Dataset 5 Experiments 6 Interactive Co-segmentation for Object-of- Interest 3D Modeling

14
Publicly available CMU-Cornell iCoseg: 38 groups 643 images ~17 im/gp Sport

15
The CMU-Cornell iCoseg Dataset Dataset Annotation: The ground-truth annotations for the dataset were manually generated by a single annotator using a labelling tool.

16
Dataset Statistics Size The histogram of the number of images in groups

17
Dataset Statistics Appearance

18
Dataset Statistics Scale

19
Outline 1 Introduction 2 iCoseg: Energy Minimization 3 iCoseg: Guiding User Scribbles 4 The CMU-Cornell iCoseg Dataset 5 Experiments 5.1 Machine Experiments 5.2 User Study 6 Interactive Co-segmentation for Object-of- Interest 3D Modeling

20
5.1 Machine Experiments

21
5.2 User Study

22
Outline 1 Introduction 2 iCoseg: Energy Minimization 3 iCoseg: Guiding User Scribbles 4 The CMU-Cornell iCoseg Dataset 5 Experiments 6 Interactive Co-segmentation for Object-of- Interest 3D Modeling

23
Interactive 3D modelling

24
3D Modeling

25
Conclusions iCoseg that co-segments all images in the group using an energy minimization framework, and an automatic recommendation system that intelligently recommends a region among all images in the group where the user should scribble next. Achieve good quality segmentations with significantly lower time and effort than exhaustively examining all cutouts.

Similar presentations

OK

Image Categorization by Learning and Reasoning with Regions Yixin Chen, University of New Orleans James Z. Wang, The Pennsylvania State University Published.

Image Categorization by Learning and Reasoning with Regions Yixin Chen, University of New Orleans James Z. Wang, The Pennsylvania State University Published.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on object-oriented programming concepts php Ppt on global marketing strategies Ppt on electricity from wastewater How to save ppt on ipad mini Ppt on ministry of corporate affairs of india File converter pdf to ppt online Ppt on australian continent Ppt on classical economics and unemployment Ppt on written business communication Ppt on eye oscillations