CSCI 631 – Foundations of Computer Vision March 15, 2016 Ashwini Imran Image Stitching

 Scope  Introduction  Image Stitching Approaches  Image Stitching Model  SIFT  RANSAC  Challenges  Future Scope Overview

To develop an image stitching algorithm which will create a panoramic image from a set of images by using feature extraction method Scope

 Image stitching is also known as mosaicking  It is a process of combining multiple pictures of the same scene into a single high resolution image  There are two techniques of achieving image stitching:  Direct Technique  Feature Based Technique Introduction

 It compares all pixels in the images with each other  Advantages:  They make optimal use of information available in image alignment  They measure contribution of every pixel in the images  Disadvantages:  Computationally complex  Time consuming  Examples:  Fourier Analysis Technique  A Unifying Framework Direct Technique

 It aims to determine similarity between images by comparing features  They establish correspondence between lines, edges, corners, points and other geometric entities  Steps in these techniques:  Feature Extraction  Registration  Blending  Advantages: Robust against scene movement, faster, ability to detect panoramas etc  Examples: SIFT, SURF, ORB etc Feature Based Technique

Image Stitching Model Image Acquisition Feature Detection and Matching Image Matching Global Alignment Blending

 Image Acquisition : it involves retrieving an image from the source  Feature Detection & Matching : it compares features between images in order to determine translation  Image Matching : it finds which image is neighbor of other image  Global Alignment : the aim is to find a set of parameters which minimize mis-registration between images  Blending : it selects final surface for the image and blends the images to form a single smooth image Image Stitching Model

The SIFT technique is one of the most robust and widely used image matching algorithm based on local features. SIFT is a feature detection and description technique. SIFT produces key point descriptors which describes the image features.

Construct Scale space DoG Locate DoG Extrema Potential feature points Filter edge Assign key points orientations Build key point descriptors Store the vector descriptor Do the matching

Gaussian Kernel is used to create Scale space. Down sample every time by a factor of 2. Take the difference of last Gaussian and store it.

Scan all DoG images obtained from previous process. Take top 2 images from the stack. Look at all neighboring points and decide the maxima and minima. For each pixel, marked as X there will be 26 points to compare.

Find the orientation of each feasible potential points. Calculate gradient dx and dy. Calculate direction theta. For each 16x16 block, we register direction in 8 different views.

It finds nearest neighbors with high probability using only a limited amount of computation. The BBF algorithm uses a modified search ordering for the k-tree algorithm so that bins in feature space are searched in order of their closest distance from the query location. It uses heap based priority queue for efficiency. The nearest neighbors are defined as the key points with minimum Euclidean Distance from the given descriptor vector. The probability that a match is correct can be determined by taking the ratio of distance from the closest neighbor to the distance of the second closest. Normally, we use 0.8 as the threshold ratio, which eliminates 90% of the false matches.

Random sample consensus ( RANSAC ) is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers. RANSAC Algorithm Repeat N times 1.Randomly select a sample Select just enough points to recover the parameters 2.Fit the model with random sample 3.See how many other points agree Best estimate is one with most agreement

Initial Matched Points

Final Matched Points

Left pyramidRight pyramid blend At low frequencies, blend slowly At high frequencies, blend quickly

 An image is often corrupted with noise during its acquisition which in turn hampers feature extraction  Image stitching on large set of data requires more time  Small scene motion such as waving tree branch and large scene motion like people moving in and out of the pictures Challenges

 Photosynth: stitching many images into one large composite image  Video Stitching Future Scope

 Ebtsam Adel Mohammad Elmogy Hazem Elbakry. “Image Stitching Based on Feature Extraction Techniques: A survey”. International Journal of Computer Applications, volume 99, August  Shikha Arya. “A Review on Image Stitching and its Different Methods”. International Journal of Advanced Research in Computer Science and Software Engineering, Volume-5, Issue-5, May  Image Stitching Wikipedia: References

Please feel free to ask any questions or concerns THANK YOU Questions & Discussion