Presentation on theme: "Image Matching and Retrieval by Repetitive Patterns Petr Doubek, Jiri Matas, Michal Perdoch and Ondrej Chum Center for Machine Perception, Czech Technical."— Presentation transcript:
Image Matching and Retrieval by Repetitive Patterns Petr Doubek, Jiri Matas, Michal Perdoch and Ondrej Chum Center for Machine Perception, Czech Technical University in Prague, Czech Republic Detection of repetitive patterns in images is a well-established computer vision problem. However, the detected patterns are rarely used in any application. A method for representing a lattice or line pattern by shift-invariant descriptor of the repeating tile is presented. The descriptor respects the inherent shift ambiguity of the tile definition and is robust to viewpoint change. Repetitive structure matching is demonstrated in a retrieval experiment where images of buildings are retrieved solely by repetitive patterns. Motivation Multiple occurrences of a local patch pose a problem to one-to- one matching algorithms (local matches are ambiguous) The presence of repeated local patches is in most cases non- accidental and therefore very distinctive Regular and even non-regular repetitive patterns give a rise to geometric constraints Repetitive Pattern and Lattice Detection Original image Detected lattice Detected tiles Zero-phased tiles Shift-Invariant Tile Representation Each repeated pattern is represented by an average appearance of a tile – a mean tile(a more complex representation possible) After discovering lattice of a repeated pattern an intrinsic shift ambiguity remains We propose two shift-invariant representations: the magnitude of Fourier coefficients of a tile zero-phase normalization: tile shifted so that phase of the first harmonic equals zero Outline of the Algorithm Detection 1.Detect repeated elements, find the lattice of the repeated pattern, rectify the lattice and calculate the mean tile 2.Compute shift invariant tile descriptors Image Matching and Retrieval 3.For each pair score with the most similar patterns 1.Detection of repeated elements. In our implementation affine covariant regions (MSERs and Hessian Affine) described by SIFT 2.Agglomerative clustering of SIFTs. Each cluster hypothesise a repeated pattern 3.For each element in the repeated pattern, find spatial nearest neighbours in each of the spatial sectors 4.Find dominant vanishing points by Hough transform and form a 2D lattice 5.Rectify lattice and divide pattern into tiles A cluster of repeated elements Nearest neighbours and neighbourhood sectors A lattice from two vanishing points (corresponding to red and green directions) TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A A A A For a pair of images i, j with sets of detected repeated patterns C i and C j similarity s k, l of two patterns k, l is computed as where and are shift invariant tile descriptions and are pairs of peaks in RGB colour histograms. Repeated Patterns Similarity Image Retrieval by Repetitive Patterns Dataset Query Top three best matches. Experiment 2 Detection and matching of repeated patterns tested on image retrieval Two publicly available datasets PSU-NRT(subset) and Pankrac+Marseille (http://cmp.felk.cvut.cz/data/repetitive)http://cmp.felk.cvut.cz/data/repetitive Top three matches for some of the queries Experiment 1 Performance of shift invariant representation Ground truth for each query G i = set of images of building i, was manually marked Tested on two datasets with ~230 images in image retrieval of about 50 buildings Conclusions Image retrieval can benefit from repeated patterns if they are detected and handled properly Proposed approach is able to detect 1D and 2D lattices under affine transformation Shift invariant descriptors addresses tile ambiguity We have shown retrieval based solely on repeated patterns, however it can be combined with standard bag-of-words retrieval approaches References T. Tuytelaars, A. Turina, and L. Van Gool, Non-combinatorial detection of regular repetitions under perspective skew, PAMI, vol.25, no.4, pp. 418-432, April 2003 P. Doubek, J. Matas, M. Perdoch and O. Chum, Detection of 2D lattice patterns of repetitive elements and their use for image retrieval, technical report, CTU-CMP-2009-16, 2009 T. K. Leung and J. Malik, Detecting, localizing and grouping repeated scene elements from an image, ECCV, 1996, pp. 546- 555 The authors were supported by Czech Science Foundation Project 102/07/1317 and by EC project FP7-ICT-247022 MASH. Example: repeated patterns of two images and their similarity p i k =(p i k; 1 ; p i k; 2 ),p j l =(p j l; 1 ; p j l; 2 ) d i k d j l Abstract 0.56 0.370.070.080.060.42 0.000.010.00 0.030.02 10x10 43px12x10 27px6x9 39px2x6 53px16x1 15px7x11 63px 12x10 57px 17x4 73px
Your consent to our cookies if you continue to use this website.