Xianyou Hou, Li-Yi Wei, Heung-Yeung Shum, Baining Guo Real-time Multi-perspective Rendering on Graphics Hardware

Teaser Real-time near object reflection off curved surface Environment mapRay tracingOur technique

Camera Projection Model Single-perspective Projection directions merge at a single point Pin-hole, orthographic, oblique, etc Supported on graphics hardware Multi-perspective Arbitrary projection directions General linear camera [Yu and McMillan ] Not directly supported on graphics hardware

Why Multi-perspective Projection? Model many natural phenomena curved reflectionrefractioncaustics may not intersect at one point!

Goal and Previous Work curved reflection near geometry fully dynamic speed Ray tracingОООХ Environment map ОХОО Image/depth sprite ООХО Mirror or gem stone ХООО [Ofek et al 1998] concavity?ОО depend on dicing Our methodООО polygon rasterization

Our Methodology Multi-perspective projection on feed-forward pipeline Similar to beam tracing Handles only planlar reflection Curved reflection via polygon rasterization (as opposed to ray tracing on GPU) Custom vertex and fragment program Non-linear rasterization

Algorithm Overview Build coarse triangle mesh Determine projection directions at mesh vertices Render each camera triangle via polygon rasterization Multi-perspective projection Custom vertex/fragment program

Parameterization Barycentric interpolation C 0 continuity p v1v1 d1d1 d d3d3 d2d2 v3v3 v2v2 d = w 1 d 1 + w 2 d 2 + w 3 d 3 w1w1 w2w2 w3w3

Rasterize one scene △ in one camera △ Vertex program: bounding triangle estimation Fragment program: ray-triangle intersection d1d1 d2d2 d3d3 v1v1 v2v2 v3v3 p11p11 p13p13 p12p12 p ij = projection of v j under d i killed shaded

Limitations and Acceleration Very brute force Rasterize every scene △ in every camera △ Bounding △ over-estimation No Zcull Acceleration object culling bounding △ culling camera tessellation level

Camera Tessellation Level Ray tracing26 (51 fps) 26x4 (63 fps)26x4x4 (5 fps) Smaller camera △ provides better bounding △ estimation faster

Performance # scene △ # camera △ fps no acceleration fps + acceleration

Applications reflectionrefraction causticsvisualization

Refraction Ray tracingOur result Only C 0 continuity

Caustics 1. Multi-perspective rendering into light point 2. Photon splatting light photon location map reflector caustic receiver

Visualization

Future Work Speed improvement Tighter bounding △ estimation Multiple reflections/refractions (i.e. beam tracing) C k continuity with k > 0 Ride with polygon rasterization! GPU is not designed for ray tracing [Purcell et al 2002] CPU ray tracing [Wald et al 2006] ?

Thank you!