Stereoscopic 3D 2013/10/30

Stereoscopic Image Transforms to Autostereoscopic Multiplexed Image Wei-Ming Chen, Chi-Hao Chiou and Sheng-Hao Jhang Computer Science and Automation Engineering (CSAE), 2011

Outline Introduction Related Work Proposed Method Experimental Results Conclusion 3

Introduction 4

3D technologies have become popular in recent years. Widely applied to movie, films and show. In early 3D vision technology: Anaglyph Polarization Shutter 5 With special glasses

Anaglyph Glass

Polarization Glass

Shutter Glass

Introduction Autostereoscopic 3D display Non-glass system Propose a new technique for generating Autostereoscopic multiplexed content. 9 Objective:

Related Work 10

Disparity Vertical Parallax Usually = 0 focus on horizontal parallax Horizontal Parallax Identify the distance of the object

Disparity retina

Image Rectification Zero Parallax Zero parallax plane

Image Rectification Simplified to one dimension - horizontal

Solution : All epipolar lines are parallel in the rectified image plane. Image Rectification

3D coordinate of real scene: Disparity to Depth f : focal length b : the length of baseline d : disparity (u 0, v 0 ) : coordinate of image center (camera intrinsic parameter matrix K) (u, v) : pixel coordinate Du Xin, Zhu Yun-fimg, "A Flexible Method for 3D Reconstruction of Urban Building", ICSP 2008 proceedings. Baseline Epipolar line

3D Reconstruction Wei-wei Ma, My-Ha Le, Kang-Hyun Jo, "3D Reconstruction and Measurement of Indoor Object Using Stereo Camera", The 6th International Forum on Strategic Technology, 2011.

Autostereoscopic 3D display Time-multiplexed Switch rapidly (left and right images) 2D & 3D : same resolution Spatial-multiplexed Parallax barrier Lenticular lenses Lower resolution for 3D 18

Autostereoscopic 3D display Parallax barrier 19 Lenticular lenses

Proposed Method 20

System Flow of Depth-map Generation 21 Stereo matching

Depth-map Generation 1) Feature point detection Use SURF algorithm [5] (based on SIFT) 2) Epipolar Geometry Matching the feature points 3) Interpolation Estimate the pixels which is not feature points 4) Graphcut Grouping the close pixels (segmentation) 22 Disparity between stereo images [5] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-Up Robust Features (SURF)," Computer Vision and Image Understanding, vol. 110, pp , 2008.

System Flow of Synthesis 23

Zero-parallax setting The most comfortably distance between user and display could be determined from Z c. Z c ( Base plane) ： 24 Z far : the highest depth map value Z near : the lowest depth map value

Pre-processing the Depth Map Mean filter Gaussian filter 25

3D-image Warping Multi-View ： Need large storage space use depth map to create virtual views Warping ： Reference view → Virtual view 26 warp

3D-image Warping Multi-View ： Need large storage space use depth map to create virtual views 27 Warping

3D-image Warping Lenticular autostereoscopic display DIBR 28 S0S0 The pixel position of the center view k View ID ( K= -4 ~ 4 ) b The distance between two eyes ZFZF Farthest distance ZNZN Nearest distance pzpz Depth value Pixels/cm (according to monitor) d Distance between eyes and screen

Issues[*]: Disparity Range Limitations of perception and technology Disparity Sensitivity More sensitive to nearby objects Disparity Gradient Disparity Velocity Temporal information 3D-image Warping 29 [*] :Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A. & Gross, M. (2010, July). Nonlinear Disparity Mapping for Stereoscopic 3D. To appear in ACM Transactions on Graphics (Proc. SIGGRAPH).

Disoccluded Regions-filling Disoccluded Regions : regions without warped pixel 30 warp Reference view Virtual view

Disoccluded Regions-filling Associated with DIBR: 31 Similar to occlusion handling

Disoccluded Regions-filling 32

Experimental Results 33

Experimental Results Disoccluded Regions-filling: 34 Previous Work Proposed

Experimental Results 35 The six warping views The six warping views

Experimental Results 36 Synthesized result

Conclusion 37

Conclusion 3D-image generation of stereo images with good 3D effect was proposed. Future : using temporal information Stereo images → Disparity map → 3D warping → Hole filling Issues: Zero-parallax setting Disparity Range / Disparity Velocity Hole filling 38

Reference Optical Design, Fabrication, and Measurement 3D Introduction and Project (Dept. of Photonics & Display Institute,National Chiao Tung University) Image Rectification (Stereo), Guido Gerig AGENCY1903 BLOG 基於 3D 顯示器格式之即時 3D 內容合成技術 ( 劉楷哲、吳其霖、黃偉豪、陳信榮、李錕、羅豐祥 )