Image Stitching II Linda Shapiro EE/CSE 576.

Slides:



Advertisements
Similar presentations
Blending and Compositing Computational Photography Connelly Barnes Many slides from James Hays, Alexei Efros.
Advertisements

Multiscale Analysis of Images Gilad Lerman Math 5467 (stealing slides from Gonzalez & Woods, and Efros)
Recap from Monday Frequency domain analytical tool computational shortcut compression tool.
Recap from Monday Fourier transform analytical tool computational shortcut.
Photo Stitching Panoramas from Multiple Images Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/28/12.
Mosaics con’t CSE 455, Winter 2010 February 10, 2010.
Photo Stitching Panoramas from Multiple Images Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/3/15.
Lecture 14: Panoramas CS4670: Computer Vision Noah Snavely What’s inside your fridge?
Announcements Project 2 Out today Sign up for a panorama kit ASAP! –best slots (weekend) go quickly...
Blending and Compositing : Rendering and Image Processing Alexei Efros.
Announcements Project 1 artifact voting ( announce later today) Project 2 out today (help session at end of class) IMPORTANT: choose Proj 2 partner.
Recognising Panoramas
Image Pyramids and Blending
1Jana Kosecka, CS 223b Cylindrical panoramas Cylindrical panoramas.
Automatic Panoramic Image Stitching using Local Features Matthew Brown and David Lowe, University of British Columbia.
Lecture 22: Structure from motion CS6670: Computer Vision Noah Snavely.
Image Stitching and Panoramas
Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending.
Announcements Project 1 artifact voting Project 2 out today panorama signup help session at end of class Guest lectures next week: Li Zhang, Jiwon Kim.
1Jana Kosecka, CS 223b Cylindrical panoramas Cylindrical panoramas with some slides from R. Szeliski, S. Seitz, D. Lowe, A. Efros,
CSCE 641 Computer Graphics: Image Mosaicing Jinxiang Chai.
Image Compositing and Blending : Computational Photography Alexei Efros, CMU, Fall 2007 © NASA.
Lecture 8: Image Alignment and RANSAC
Announcements Project 1 Grading session this afternoon Artifacts due Friday (voting TBA) Project 2 out (online) Signup for panorama kits ASAP (weekend.
Image Pyramids and Blending : Computational Photography Alexei Efros, CMU, Fall 2005 © Kenneth Kwan.
Image Stitching II Ali Farhadi CSE 455
COS 429 PS3: Stitching a Panorama Due November 4 th.
Announcements Project 1 artifact voting Project 2 out today (help session at end of class)
Image Pyramids and Blending
Image Stitching Ali Farhadi CSEP 576 Several slides from Rick Szeliski, Steve Seitz, Derek Hoiem, and Ira Kemelmacher.
Announcements Midterm due Friday, beginning of lecture Guest lecture on Friday: Antonio Criminisi, Microsoft Research.
Image Stitching Ali Farhadi CSE 455
Seamless image-based texture atlases using multi-band blending Cédric AllèneJean-Philippe PonsRenaud Keriven Université Paris-Est – Ecole des Ponts – CERTIS.
By Yevgeny Yusepovsky & Diana Tsamalashvili the supervisor: Arie Nakhmani 08/07/2010 1Control and Robotics Labaratory.
Prof. Alex Berg (Credits to many other folks on individual slides)
Image Stitching Shangliang Jiang Kate Harrison. What is image stitching?
Mosaics Today’s Readings Szeliski, Ch 5.1, 8.1 StreetView.
Image Stitching Ali Farhadi CSE 576 Several slides from Rick Szeliski, Steve Seitz, Derek Hoiem, and Ira Kemelmacher.
CS4670 / 5670: Computer Vision KavitaBala Lecture 17: Panoramas.
10/19/10 Image Stitching Computational Photography Derek Hoiem, University of Illinois Photos by Russ Hewett.
Mosaics part 3 CSE 455, Winter 2010 February 12, 2010.
CS 534 Homework #4 References. Cylindrical Projection Alpha Blending.
Image Stitching II Linda Shapiro CSE 455. RANSAC for Homography Initial Matched Points.
Computer vision. Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids DFT - Discrete Fourier transform.
Mosaics Today’s Readings Szeliski and Shum paper (sections 1 and 2, skim the rest) –
COS 429 PS3: Stitching a Panorama Due November 10 th.
Image Stitching Computer Vision CS 691E Some slides from Richard Szeliski.
Hebrew University Image Processing Exercise Class 8 Panoramas – Stitching and Blending Min-Cut Stitching Many slides from Alexei Efros.
Recall: Gaussian smoothing/sampling G 1/4 G 1/8 Gaussian 1/2 Solution: filter the image, then subsample Filter size should double for each ½ size reduction.
Image Stitching II Linda Shapiro EE/CSE 576.
CS4670 / 5670: Computer Vision Kavita Bala Lecture 20: Panoramas.
Image Blending : Computational Photography
Interest Points EE/CSE 576 Linda Shapiro.
COSC579: Image Align, Mosaic, Stitch
Image Stitching Computational Photography
Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi.
Multiscale Analysis of Images
Idea: projecting images onto a common plane
Announcements Project 2 out today (help session at end of class)
Image Stitching II Linda Shapiro EE/CSE 576.
Image Stitching II Linda Shapiro CSE 455.
Image Stitching Computer Vision CS 678
Announcements Project 1 Project 2 Vote for your favorite artifacts!
Announcements Project 1 artifact voting Project 2 out today
Photometric Processing
CS 534 Homework #4 References
Image Stitching Linda Shapiro ECE/CSE 576.
CS Homework #4 References
Image Stitching II Linda Shapiro ECE P 596.
Image Stitching Linda Shapiro ECE P 596.
Presentation transcript:

Image Stitching II Linda Shapiro EE/CSE 576

RANSAC for Homography Initial Matched Points

RANSAC for Homography Final Matched Points

RANSAC for Homography

Image Blending What’s wrong?

Feathering 1 + ramp =

Effect of window (ramp-width) size 1 left 1 right

Effect of window size 1 1

Good window size What can we do instead? 1 What can we do instead? “Optimal” window: smooth but not ghosted Doesn’t always work...

Pyramid blending Create a Laplacian pyramid, blend each level apple orange Create a Laplacian pyramid, blend each level Burt, P. J. and Adelson, E. H., A Multiresolution Spline with Application to Image Mosaics, ACM Transactions on Graphics, 42(4), October 1983, 217-236. http://persci.mit.edu/pub_pdfs/spline83.pdf

Forming a Gaussian Pyramid Start with the original image G0 Perform a local Gaussian weighted averaging function in a neighborhood about each pixel, sampling so that the result is a reduced image of half the size in each dimension. Do this all the way up the pyramid Gl = REDUCE(Gl-1) Each level l node will represent a weighted average of a subarray of level l.

Making the Laplacians We want to subtract each level of the pyramid from the next lower one. But they are different sizes! In order to do the subtraction, we perform an interpolation process. We interpolate new samples between those of a given image to make it big enough to subtract. The operation is called EXPAND.

The Laplacian Pyramid Gaussian Pyramid Laplacian Pyramid - = - = - =

To blend two images, We’ll combine two Laplacian pyramids Laplacian Pyramid LA Laplacian Pyramid LS Laplacian Pyramid LB to be filled in

Forming the New Pyramid Laplacian pyramids LB and LB are constructed for images A and B, respectively. A third Laplacian pyramid LS is constructed by copying nodes from the left half of LA to the corresponding nodes of LS and nodes from the right half of LB to the right half of LS. Nodes along the center line are set equal to the average of corresponding LA and LB nodes

Using the new Laplacian Pyramid Use the new Laplacian pyramid with the reverse of how it was created to create a Gaussian pyramid. The lowest level of the new Gaussian pyramid gives the final result.

Laplacian level 4 Laplacian level 2 Laplacian level left pyramid with the ramp function Laplacian level left pyramid right pyramid blended pyramid

Pyramid blending Create a Laplacian pyramid, blend each level apple orange Create a Laplacian pyramid, blend each level Burt, P. J. and Adelson, E. H., A Multiresolution Spline with Application to Image Mosaics, ACM Transactions on Graphics, 42(4), October 1983, 217-236. http://persci.mit.edu/pub_pdfs/spline83.pdf

Multiband blending (IJCV 2007) Laplacian pyramids Compute Laplacian pyramid of images and mask Create blended image at each level of pyramid Reconstruct complete image 3 different frequency levels of blending; mask is blurred in lower levels

Blending comparison (IJCV 2007)

Poisson Image Editing For more info: Perez et al, SIGGRAPH 2003 http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf

Alpha Blending Encoding blend weights: I(x,y) = (αR, αG, αB, α ) Optional: see Blinn (CGA, 1994) for details: http://ieeexplore.ieee.org/iel1/38/7531/00310740.pdf?isNumber =7531&prod=JNL&arnumber=310740&arSt=83&ared=87&arAut hor=Blinn%2C+J.F. I2 A: render as black Encoding blend weights: I(x,y) = (αR, αG, αB, α ) color at p = Implement this in two steps: 1. accumulate: add up the (α premultiplied) RGB values at each pixel 2. normalize: divide each pixel’s accumulated RGB by its α value

Choosing seams dynamic program to find seam along well-matched regions Illustration: http://en.wikipedia.org/wiki/File:Rochester_NY.jpg

Gain Compensation: Getting rid of artifacts Simple gain adjustment Compute average RGB intensity of each image in overlapping region Normalize intensities by ratio of averages

Blending Comparison

Recognizing Panoramas Brown and Lowe 2003, 2007 Some of following material from Brown and Lowe 2003 talk

Recognizing Panoramas Input: N images Extract SIFT points, descriptors from all images Find K-nearest neighbors for each point (K=4) For each image Select M candidate matching images by counting matched keypoints (m=6) Solve homography Hij for each matched image

Recognizing Panoramas Input: N images Extract SIFT points, descriptors from all images Find K-nearest neighbors for each point (K=4) For each image Select M candidate matching images by counting matched keypoints (m=6) Solve homography Hij for each matched image Decide if match is valid (ni > 8 + 0.3 nf ) # keypoints in overlapping area # inliers

Recognizing Panoramas (cont.) (now we have matched pairs of images) Make a graph of matched pairs Find connected components of the graph

Finding the panoramas

Finding the panoramas

Recognizing Panoramas (cont.) (now we have matched pairs of images) Find connected components For each connected component Solve for rotation and f Project to a surface (plane, cylinder, or sphere) Render with multiband blending

Finding the panoramas