Presentation on theme: "Boundary Preserving Dense Local Regions"— Presentation transcript:
1 Boundary Preserving Dense Local Regions Jaechul Kim and Kristen GraumanUniv. of Texas at Austin
2 Local feature detection A crucial building block for many applicationsImage retrievalObject recognitionImage matchingKey issue:How to detect local regions for feature extraction?
3 Related work Interest point detectors Dense sampling e.g., Matas et al. (BMVC 02), Jurie and Schmid (CVPR 04), Mikolajczyk and Schmid (IJCV 04)Dense samplinge.g., Nowak et al. (ECCV 06)Segmented regions and Superpixelse.g., Ren and Malik (ICCV 03) , Gu et al. (CVPR 09),Todorovic and Ahuja (CVPR 08),Malisiewicz and Efros (BMVC 07), Levinshtein et al. (ICCV 09)Hybride.g., Tuytelaars (CVPR 10), Koniusz and Mikolajczyk (BMVC 09)
4 What makes a good local feature detector? Desired properties:- Repeatable- Boundary-preserving- Distinctively shapedExisting methods lack one or more of these criteria, e.g.,Lack repeatabilityLack distinctive shape,straddle boundariesSegmentsDense samplingInterest points
5 Our idea: Boundary Preserving Local Regions (BPLRs) Boundary preserving, dense extractionSegmentation-driven feature sampling and linkingRepeatable local features capturing objects’ local shapes
6 Approach: Overview Sampling elements Linking elements Initial elements for each segment are sampled based on distance transform of the segmentA segmentSampled elementsLinking elementsA single graph structure reflecting main shapes and segment layoutMin. spanning treeGrouping elementsGrouping neighboring elements into BPLRNeighbor elementsBPLR
7 Approach: Sampling Input image Segment Sampled elements LinkingGroupingZoom-in viewxAn ”element”xInput imageSegmentSampled elementsfrom “all” segmentsDistance transformDense regular gridSampled elements
9 Role of spanning tree linkage SamplingRole of spanning tree linkageLinkingGroupingMin spanning tree prefers to link closer elements+Multiple samplingDue to distance transform-based sampling same-segment elements more likely linkedDue to multiple segmentations elements in overlapping segments more likely linked
10 Approach: Grouping Intersection of topology and Euclidean neighbor SamplingApproach: GroupingLinkingGroupingDescriptorIntersection of topology and Euclidean neighborReference element’s locationBPLRIntersection of topology and Euclidean neighborReference element’s locationNeighbor elementsReference element’s locationZoom-in viewReference element’s locationEuclidean neighbor elements’ locationEuclidean neighborReference element’s locationTopological neighbor elements’ locationTopological neighborExample detections of BPLRs(Subset shown for visibility)Reference element’s location
12 Experiments 20-200 segments ~7000 BPLRs in 400 x 300 image Tasks: 2-5 seconds to extract BPLRs per an imagePHOG + gPb descriptor usedTasks:RepeatabilityLocalizationForeground segmentationObject classificationBaselines:Dense sampling (+ SIFT)MSER (+ SIFT) Semi-local regions (+ SIFT) [2,3]Segmented regions (+ PHOG) Superpixels  Matas et al., BMVC 02.  Quack et al., ICCV 07. Lee and Grauman, IJCV 09.  Arbelaez et al., CVPR 09. Ren and Malik, ICCV 03.
13 Example feature extractions Proposed BPLRs(Subset shown for visibility)SegmentedregionsSuperpixelsInterest regions(MSERs)Dense sampling
14 Repeatability for object categories Bounding Box Hit Rate – False Positive Rate [Quack et al. 2007]ApplelogoTest imageGiraffeBottleTrain imagesSwanMugTrue matchFalse positiveComparison to baseline region detectors on ETHZ shape classes
15 Localization accuracy Bounding Box Overlapping Score – RecallApplelogoGiraffeBottleCompute overlapping score by projecting the training exemplar’s bounding box into the test imageSwanMugComparison to baseline region detectors on ETHZ shape classes
16 Localization accuracy Test imageDatabase images with best matches to test BPLRs
17 Foreground segmentation Replacing superpixels with BPLRs in GrabCut segmentationApproachAccuracy(%)BPLR + GrabCut (Ours)85.6Superpixel + GrabCut81.5Superpixel ClassCut (Alexe et al., ECCV 10)83.6Superpixel Spatial Topic Model (Cao et al., ICCV 07)67.0Foreground segmentation in Caltech-28 dataset
18 Object classification Nearest-neighbor results on Caltech-101 benchmarkFeatureAccuracy(%)BPLR + PHOG (Ours)61.1Dense + SIFT55.2Segment + PHOG37.6Dense + PHOG27.9Comparison of features using the same Naïve Bayes NN [Boiman et al. 2008] classifier.
19 Conclusion Dense local detector that preserves object boundaries Capture object’s local shape in a repeatable mannerFeature sampling and linking driven by segmentationGeneric bottom-up extractionCode available: