Presentation on theme: "From Interactive to Semantic Image Segmentation Varun Gulshan Supervisors: Prof. Andrew Blake Prof. Andrew Zisserman 20 Jan 2012."— Presentation transcript:
From Interactive to Semantic Image Segmentation Varun Gulshan Supervisors: Prof. Andrew Blake Prof. Andrew Zisserman 20 Jan 2012
Two segmentation tasks car building person tree sky road bench object background Interactive segmentationSemantic segmentation
Interactive segmentation Semantic segmentation Thesis Flow Chapter 3: Texture Features Low level cues Chapter 4: Star convexity Mid level cues Chapter 5: Segmenting humans Bounding box interaction + Top down cues Chapter 6: Superpixel based classification Fully automatic segmentation
Chapter 3: Features for interactive segmentation Low level texture features for improving interactive segmentation methods.
Texture features Pure Texture Feature (L-shape): Texture + Gray Feature (L-shape):
Texture features Pure Texture Feature (Plus-shape): Texture + Gray Feature (Plus-shape):
Camouflage Image Dataset Introduced a dataset of 50 Camouflage images, to demonstrate the power of texture features.
Quantitative evaluation +14% +21% +7%+4% Gray RGB Huge gain in accuracy obtained using texture features on top of gray scale images. Significant improvement on top of RGB images.
Chapter 4: Star Convexity and Extensions Mid level shape constraints for reducing user effort in interactive segmentation systems.
Chapter 4: Star convexity Single Star Multiple Stars Geodesic Star
Robot user evaluation Updated segmentation Biggest connected component Initial brush strokes Segmentation output with current interaction Error segmentation False positive False negative New Brush Stroke Centre of connected component New Brush Stroke New brush stroke placed Process is repeated upto 20 strokes Segmentation after 20 strokes
Robot user evaluation MethodSP-IGSP-SIGSP-LIGBJRWGSCseq Effort Our method takes least effort
Chapter 5: Learning to segment humans Using top down cues to segment specific object categories.
Segmenting humans Bounding box (given/detected) Top down HOG prediction Bottom up refinement
Kinect Data Acquisition RGB imageKinect scene labelsCleaned up Ground truth Dataset of roughly 3500 images acquired using the Kinect
Top down learning Local ImageLocal HOGLocal mask Classifier trained to predict segmentation masks for local windows based on their HOG descriptor.
Bottom up refinement Top down segmentation Local Color model window Local color model unaries Final segmentation …..
Chapter 6: Semantic segmentation Fully automatic segmentation based upon learning from multiple superpixelisations.
Combing multiple superpixelisations Various methods to learn from multiple superpixelisations explored: 1. Avg-Indep 2. Avg-Union 3. LPβ-Indep 4. IofR-Joint GlobalPbVekslerQuickShift