1. Xianyou Hou, Li-Yi Wei, Heung-Yeung Shum, Baining Guo Real-time Multi-perspective Rendering on Graphics Hardware

2. Teaser Real-time near object reflection off curved surface Environment mapRay tracingOur technique

3. Camera Projection Model Single-perspective Projection directions merge at a single point Pin-hole, orthographic, oblique, etc Supported on graphics hardware Multi-perspective Arbitrary projection directions General linear camera [Yu and McMillan 2004 - 2005] Not directly supported on graphics hardware

4. Why Multi-perspective Projection? Model many natural phenomena curved reflectionrefractioncaustics may not intersect at one point!

5. Goal and Previous Work curved reflection near geometry fully dynamic speed Ray tracingОООХ Environment map ОХОО Image/depth sprite ООХО Mirror or gem stone ХООО [Ofek et al 1998] concavity?ОО depend on dicing Our methodООО polygon rasterization

6. Our Methodology Multi-perspective projection on feed-forward pipeline Similar to beam tracing Handles only planlar reflection Curved reflection via polygon rasterization (as opposed to ray tracing on GPU) Custom vertex and fragment program Non-linear rasterization

7. Algorithm Overview Build coarse triangle mesh Determine projection directions at mesh vertices Render each camera triangle via polygon rasterization Multi-perspective projection Custom vertex/fragment program

8. Parameterization Barycentric interpolation C 0 continuity p v1v1 d1d1 d d3d3 d2d2 v3v3 v2v2 d = w 1 d 1 + w 2 d 2 + w 3 d 3 w1w1 w2w2 w3w3

9. Rasterize one scene △ in one camera △ Vertex program: bounding triangle estimation Fragment program: ray-triangle intersection d1d1 d2d2 d3d3 v1v1 v2v2 v3v3 p11p11 p13p13 p12p12 p ij = projection of v j under d i killed shaded

10. Limitations and Acceleration Very brute force Rasterize every scene △ in every camera △ Bounding △ over-estimation No Zcull Acceleration object culling bounding △ culling camera tessellation level

11. Camera Tessellation Level Ray tracing26 (51 fps) 26x4 (63 fps)26x4x4 (5 fps) Smaller camera △ provides better bounding △ estimation faster

12. Performance # scene △ # camera △ fps no acceleration fps + acceleration 8001004.520.1 12001003.614.6 16001002.19.2 4002561.56 8002560.52.1 12002560.351.8 4005120.82.5 8005120.31.0

13. Applications reflectionrefraction causticsvisualization

14. Refraction Ray tracingOur result Only C 0 continuity

15. Caustics 1. Multi-perspective rendering into light point 2. Photon splatting light photon location map reflector caustic receiver

16. Visualization

17. Future Work Speed improvement Tighter bounding △ estimation Multiple reflections/refractions (i.e. beam tracing) C k continuity with k > 0 Ride with polygon rasterization! GPU is not designed for ray tracing [Purcell et al 2002] CPU ray tracing [Wald et al 2006] ?

18. Thank you!