1. Exploiting Homography in Camera-Projector Systems Tal Blum Jiazhi Ou Dec 11, 2003 [Sukthankar, Stockton & Mullin. ICCV-2001]

2. Road Map Introduction Introduction Projector-Camera Homography Projector-Camera Homography Automatic Keystone Correction Automatic Keystone Correction Lines Detection Lines Detection Demo Demo Conclusion Conclusion

3. Introduction The Situation: The Situation: A presentation system with a computer and a projector The Problem: The Problem: The projected image is warped if the projector is not aligned manually The Solution: The Solution: Automatic keystone correction using a camera The Assumption The Assumption Uncalibrated camera, uncalibrated projector, we know the resolution of the screen

4. Projector-Camera Homography Presumption: Points on a plane (board) Given, we want to estimate H:,

5. Projector-Camera Homography Linear Least-Squares: Construct a 2N*9 matrix (N>=4): h equals to the eigenvector of L’*L corresponding to the smallest eigenvalue

6. Automatic Keystone Correction 1. Compute Projector-Camera Homography 2. Compute World-Camera Homography 5. Compute World-Image Homography Overview: 3. Compute World-Projector Homography 4. Compute New Projected Area on Board 6. Compute Projector-Image Homography 7. Warp Image

7. Compute Projector- Camera Homography P1P1 P2P2 P3P3 P4P4 If we know p 1, p 2, p 3, p 4, we can can estimate H 1 :

8. Compute World-Camera Homography If we know p 1, p 2, p 3, p 4, we can can estimate H 2 : P1P1 P2P2 P3P3 P4P4

9. Compute World-Projector Homography

10. Automatic Keystone Correction 1. Compute Projector-Camera Homography H 1 2. Compute World-Camera Homography H 2 Overview: 3. Compute World-Projector Homography H 3 4. Compute New Projected Area on Board

11. Compute New Projected Area on Board 1. Find old projected area:

12. Compute New Projected Area on Board 2. Find a largest rectangle in the old projected area:

13. Compute World-Image Homography Now we know we can can estimate H 4 :

14. Compute Projector-Image Homography

15. Automatic Keystone Correction 1. Compute Projector-Camera Homography H 1 2. Compute World-Camera Homography H 2 5. Compute World-Image Homography H 4 Overview: 3. Compute World-Projector Homography H 3 4. Compute New Projected Area on Board 6. Compute Projector-Image Homography H 5 7. Warp Image: For each pixel in projector, find the corresponding pixel in the image using H 5

16. Road Map Introduction Introduction Projector-Camera Homography Projector-Camera Homography Automatic Keystone Correction Automatic Keystone Correction Lines Detection Lines Detection Demo Demo Conclusion Conclusion

17. Lines Detection We used our implementation for the lines detection We used our implementation for the lines detection Problems in lines detection include: Problems in lines detection include: –Noise due to low quality camera –Need to be invariant to different room settings and different lighting conditions –The camera might be in different distances from the screen.

18. Lines Detection implementation Stages Stages –Brightness Normalization –Canny edge detection for k=1 to 4 Compute the parameter distribution Smooth the parameter space Find the point in the k’th parameter space (R_k,Theta_k) that has the maximal value and satisfy constraints. Remove the points belonging to the lines found so far from the parameter distribution end

19. Brightness Normalization Adjusting the intensity by linear transformation so that the intensity range would be [0,1] Adjusting the intensity by linear transformation so that the intensity range would be [0,1]

20. Canny edge detection

21. For detecting the projection coordinates We use the difference of an image with a white projection and an image without it. We use the difference of an image with a white projection and an image without it. The canny image is much cleaner & easier to deal with. The canny image is much cleaner & easier to deal with.

22. Counting Over Parameter Space Lines are represented as (R,Theta) Lines are represented as (R,Theta) Count for each line how many points go through it Count for each line how many points go through it Smooth with a Gaussian kernel Smooth with a Gaussian kernel –Depends on distance from the center Sampling problems Sampling problems –Lines more densely sampled near the center –Solution: Representing the points relative to the center point & Sample more densely

23. Representation Problems X Y

24. Choosing 4 lines Iteratively choose 4 lines Iteratively choose 4 lines Order the best lines by their counts Order the best lines by their counts Choose the best line that satisfy constraints Choose the best line that satisfy constraints –Constraints include that the intersections are within the image & that each line has exactly 2 intersection with the other lines Weighting lines with different angles differently to correct for vertical lines Weighting lines with different angles differently to correct for vertical lines

25. Finding the intersection

26. Road Map Introduction Introduction Projector-Camera Homography Projector-Camera Homography Automatic Keystone Correction Automatic Keystone Correction Lines Detection Lines Detection Demo Demo Conclusion Conclusion

27. Conclusion We built a presentation system that corrects keystone automatically We built a presentation system that corrects keystone automatically We exploited camera-projector homography We exploited camera-projector homography We implemented our own line detection algorithm We implemented our own line detection algorithm The authors also use this homography to define virtual buttons on the projector screen The authors also use this homography to define virtual buttons on the projector screen

28. Thank You! A502 Newell-Simon Hall {blum,jiazhiou}@cmu.edu