1. Computational Photography: Image Mosaicing Jinxiang Chai

2. Outline Image registration - feature-based registration - pixel-based registration Image mosaicing

3. Required Readings Section 9.1, Szeliski book. Section 9.3.1 recognizing panoramas [website]recognizing panoramaswebsite

4. virtual wide-angle camera Mosaics: Stitching Image Together

5. Mosaic Procedure Basic Procedure –Take a sequence of images from the same position Rotate the camera about its optical center –Compute transformation between second image and first –Transform the second image to overlap with the first –Blend the two together to create a mosaic –If there are more images, repeat

6. Image Mosaic real camera synthetic camera Can generate any synthetic camera view as long as it has the same center of projection! Is a pencil of rays contains all views

7. Image Re-projection mosaic PP The mosaic has a natural interpretation in 3D –The images are reprojected onto a common plane –The mosaic is formed on this plane –Mosaic is a synthetic wide-angle camera

8. Issues in Image Mosaic mosaic PP How to relate two images from the same camera center? - image registration How to re-project images to a common plane? - image warping ? ?

9. Image Mosaicing Geometric relationship between images

10. Image Mosaicing Geometric relationship between images –Use 8-parameter projective transformation matrix

11. Image Mosaicing Geometric relationship between images –Use 8-parameter projective transformation matrix –Use a 3D rotation model (one R per image) Derive it by yourself!

12. Image Mosaicing Geometric relationship between images –Use 8-parameter projective transformation matrix –Use a 3D rotation model (one R per image) Register all pairwise overlapping images –Feature-based registration –Pixel-based registration Chain together inter-frame rotations

13. Image Stitching Stitch pairs together, blend, then crop

14. Image Stitching A big image stitched from 5 small images

15. Panoramas What if you want a 360  field of view? mosaic Projection Cylinder

16. Cylindrical Panoramas Steps –Re-project each image onto a cylinder –Blend –Output the resulting mosaic

17. Cylindrical Projection –Map 3D point (X,Y,Z) onto cylinder X Y Z unit cylinder

18. Cylindrical Projection –Map 3D point (X,Y,Z) onto cylinder X Y Z unwrapped cylinder –Convert to cylindrical coordinates unit cylinder

19. Cylindrical Projection –Map 3D point (X,Y,Z) onto cylinder X Y Z unwrapped cylinder –Convert to cylindrical coordinates cylindrical image –Convert to cylindrical image coordinates s defines size of the final image unit cylinder

20. Cylindrical Panoramas mosaic Projection Cylinder A B Cannot map point A to Point B without knowing (X,Y,Z)

21. Cylindrical Panoramas mosaic Projection Cylinder B C A But we can map point C (images) to Point B.

22. Cylindrical Warping Given focal length f and image center (xc,yc) X Y Z (X,Y,Z)(X,Y,Z) (sin ,h,cos  )

23. Cylindrical Panoramas f = 180 (pixels) Map image to cylindrical or spherical coordinates –need known focal length Image 384x300f = 380f = 280

24. Cylindrical Panorama 3D rotation registration of four images taken with a handheld camera.

25. Cylindrical Panorama

26. Recognizing Panoramas A fully automatic 2D image stitcher system Input Images Output panorama #1

27. Recognizing Panoramas A fully automatic 2D image stitcher system Input Images Output panorama #2

28. Recognizing Panoramas A fully automatic 2D image stitcher system Input Images Output panorama #3

29. Recognizing Panoramas A fully automatic 2D image stitcher system Input Images - How to recognize which images can be used for panoramas? - How to stitch them automatically?

30. Recognizing Panoramas A fully automatic 2D image stitcher system

31. Recognizing Panoramas A fully automatic 2D image stitcher system - Image matching with SIFT featuresSIFT - For every image, find the M best images with RANSAC - Form a graph and find connected component in the graph - Stitching and blending.

32. Recognizing Panoramas

38. Outline Image registration Image mosaicing