1. A Multicamera Setup for Generating Stereo Panoramic Video Tzavidas, S., Katsaggelos, A.K. Multimedia, IEEE Transactions on Volume: 7, Issue:5 Publication Year: 2005 Professor: Jar - Ferr Yang Presenter: Kuan - Hsien - Liu

2. Outline Introduction Description of Proposed Stereo Panoramic Video Setup Analysis of Proposed Stereo Panoramic Video Setup(tradeoff) Disparity Variation Experimental Results Conclusion

3. Introduction We present a new multi-camera setup that is capable of creating the desired views (360- degree FOV with horizontal disparity) at video rates.

4. Multiple viewpoint – Different viewpoint for different viewing direction Circular Projections for Stereo Panorama Circular projections: world points P1 and P2 are projected onto the image surface by using the rays tangent to the viewing circle. Every world point P is pictured through two different viewpoints for the left and right panoramic images. Viewpoint R is used for the construction of the right panoramic image and viewpoint L for the construction of the left. Circular ProjectionCircular + Stereo

5. Rotating the regular camera and collecting small strips from the left part of the images produced by the regular camera, we construct the panoramic image for the right eye, and the left eye is Similarly. The width of the strips is a function of the number of the cameras in the rig. In a practical scenario, we only have a limited number of images available, which results in a relatively large width of the strips (tens or hundreds of pixels).  Project image to cylindrical surface before taking the strip.

6. Description of Proposed Stereo Panoramic Video Setup Consist of multiple cameras mounted on a rig with fixed geometry. We employ circular projections for the construction of each frame in the stereo panoramic video sequence. Taking the appropriate strips from the left and right part of the images and pasting them together. Moving part

7. ANALYSIS of PROPOSED STEREO PANORAMIC VIDEO SETUP A. Size of the Rig and Perspective Correction The circumference number of cameras We assume that the rig is a regular polygon, each camera occupies one side of this polygon. W: width of physical camera N: number of cameras in the rig R: radius of the rig , This setup’s key point is ( N, W )

8. B. “Blind” Areas Limited number of cameras  Some parts of the environment are not imaged by any of the cameras in the rig. Direction of view is the ray tangent to the viewing circle is defined by the angle How far the strip away from the central line of image?

9. Before taking the strips, each image is warped by performing a cylindrical projection, We obtain the strip by taking the part of the projected on the cylinder image that corresponds to angle Ω.

10. Line represents the leftmost boundary of the part of the environment that is projected onto the strip by camera1  any world points lying to the left of this line will not be projected on the strip by camera 1. Line represents the rightmost boundary of the part of the environment that is projected onto the strip by camera 2  any world points lying to the right of this line will not be projected on the strip by camera 2. 左邊界 右邊界

11. Referring to above analysis, there are three possible values for the angle Ω Ω < Ө/2: lines and intersect at a point in the lower half of the plane, the two lines move further apart thus leaving a growing part of the scene uncovered Ω = Ө/2: lines and become parallel, their slopes become equal ( ), so there are no world points that projected on both strips.  The two successive strips have no overlapping regions. Ω > Ө/2: lines and intersect at a point in the upper half of the plane. Therefore, the two successive strips overlap.  The best? Not really! We are interested in creating panoramic images at video rates and we want to avoid computationally expensive steps. So we use Ω = Ө/2

12. Because Ω = Ө/2, so there is a blind area between strips. Therefore, the distance between two parallel lines: “Blind” Areas

13. A setup is valid or not? In the previous:,

14. The increase in the number of cameras does not reduce the width of blind strips. For smaller cameras, the resulting blind strip width is smaller. N=21

15. C. Disparity Variation In actual Setup, the symmetrical construction is destroyed due to the limited cameras.  The main consequence of such lack of symmetry is a change in the disparity of objects in the resulting stereo panorama Ideal case Actual case

16. Keep the maximum disparity variation as a representative value that describes the performance with respect to disparity of this particular setup. This change in disparity between ideal and actual case is defined as disparity variation

17. Adding more cameras, beyond this critical number, does not improve the performance of the rig For a given number of cameras, there is a clear gain in going to cameras with smaller physical dimensions the maximum disparity variation is actually decreasing for smaller M when N is constant. We also observe in Fig. 13 that the maximum disparity variation even in the best case is quite significant (20% in the best case, almost 50% in the worst).

18. Is the Disparity variation likely to produce a panorama that would uncomfortable for a human viewer? It is known that disparity is a cue that is used by the human eye to judge relative depths only. when viewing a certain scene the eyes of the observer fixate at a certain depth, and use disparities to judge relative depths only for small depth differences with respect to the fixation point. We saw earlier that the disparity variation is not the same for different values of the depth. However, for a small range of depths we can assume that all corresponding disparities change approximately the same with respect to the ideal. In other words, their relative sizes remain approximately the same.

19. Disparity Variation The disparity at the same depth Z but at different directions The angular disparity of P’= We calculate the disparity 2ω and the percentage change with respect to the disparity for offset = 0, which we call 2ω0(ω0 is given by (12))

20. Smaller values of W tend to produce smaller disparity variation. An increasing N tends to reduce the percent disparity variation. AS Z increases, the distance between the different curves becomes smaller. The minimum number of cameras needed is 21, and the resulting maximum disparity variation is less than 2%.  the lack of symmetry of an actual setup produces some variation in disparity but this variation is not likely to produce any visible effects in the resulting panoramic image.

21. Experimental Results Virtual environment One frame in the panoramic video

22. N=21 W=42 (mm) N=32 W=42 (mm) N=43 W=16 (mm)

23. Conclusion The Important parameters defining a setup – Number of cameras & physical width of camera The minimum number of camera is the optimal choice (given a type of camera, ex: FOV & physical dimensions) The size of camera becomes smaller – Small rig – Reduces the blind area – Smaller disparity variation This setup will produced lots of data – New compression tech need to be developed for storage and transmission.

24. Thanks for your attention!

25. 附錄 : Circular projection For stereo panorama we develop a special type of multiple viewpoint projections, where both the left-eye image share the same cylindrical surface. To enable stereo perception, the left-eye and the right-eye are located on viewing circle. The viewing direction is on the line tangent to the viewing circle.