1. Adaptive Streaming and Rendering of Large Terrains Raphaël Lerbour Advisors: Kadi Bouatouch (IRISA) Jean-Eudes Marvie (THOMSON)

2. 2 Terrain rendering 2D maps of elevation and color samples 3D applications: games, GPS, virtual tourism…

3. 3 Objectives Render large terrain datasets – Support huge maps (dozens of GB) – Remote loading With good interactivity – Unpredicted user viewpoint moves – Speed requirements (e.g. 25 frames per second) – Real-time hardware 3D rendering On multiple target devices and networks – Desktop PCs, handhelds… – Cannot transmit or display entire dataset

4. Solutions Network limitations: adaptive client-server streaming – Progressive loading guided by rendering needs – Available subset of dataset is rendered while loading Graphics hardware limitations: adaptive rendering – 3D mesh simplification (data selection) 4 File server Server database Network Streaming controller Partial database ServerClient

5. 5 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

6. 6 Related work Commercial software – Google Earth, NASA World Wind… – Flight simulators Continuous levels of detail (LOD) [Lindstrom et al. 1996, Duchaineau et al. 1997, Hoppe 1998] – Focus on reducing mesh complexity at all costs – Every data sample is considered at each frame – Designed for slow 3D rendering hardware Alternative approaches – Clipmap [Losasso et al. 2004] – Projective grid [Dachsbacher et al. 2004] Google Earth NASA World Wind

7. Blocks with discrete levels of detail [De Boer 2000, Pouderoux et al. 2005, Schneider et al. 2006] – Map subdivided into fixed-size blocks – Per-block selection between predefined LODs – Blocks can be loaded independently – All blocks needed to render entire map Hierarchies of blocks [Levenberg 2002, Cignoni et al. 2003, Livny et al. 2007] – All blocks have the same resolution – Blocks organized in a tree – Changing LOD requires changing tree level – Tree can be loaded progressively Related work 7 Split

8. Discussion Related work – Few solutions support both streaming and rendering efficiently – Some are based on outdated considerations – Few support planetary terrains and solve corresponding issues Our approach – Unify adaptive data streaming and selection Same data structure and adaptive schemes, suited for both – But separate from 3D rendering Generic streaming and selection for large 2D maps, then application to 3D terrains – Adapt to any database size and hardware capabilities Favor speed, avoid data redundancy, support planetary terrains – Offer a complete and efficient solution 8

9. 9 Overview of our work Generic adaptive solution – Streaming and rendering of sample maps Application to 3D terrain rendering – Support of planetary terrains Offline preprocessing steps with support for huge maps File server Server database Network Adaptive streaming and selection Partial client database 3D conversion and rendering Preprocessing Original database

10. 10 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

11. 11 Hierarchy of square blocks [Levenberg et al. 2002] – Can be progressively loaded as a tree, starting with the root – Hierarchical block selection minimize amount of rendered blocks Blocks have sets of regular levels of detail (LOD) [De Boer 2000] – Adaptive LOD selection minimize amount of structure operations Data structure: base

12. 12 Data structure: new properties No data redundancy: less to store and transmit – LODs of a block share data (common sample grid) – Parent and children share one LOD (local copy when split/merge) New LOD: samples interleaved between existing ones – Possible to render a block with not all LODs loaded – Possible to render a block and load one of its LODs in parallel

13. 13 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

14. 14 Measure of importance The base for adaptivity in both streaming and rendering – Importance represents desired quality for a block – Thresholds to select LODs and trigger update requests – Defines priority for triggered requests Any factors may be used – Based on application, nature of the data – Typical factors for 3D terrain rendering: Distance from viewpoint Surface area Roughness

15. 15 Adaptive streaming Pre-computed server database for minimal activity – Only one file read per LOD request – File position for any LOD known from previous LOD Data are loaded as-is – Conversion on the client, in parallel with rendering – Possible quantization, pre-compression – Same server and data with any client type We always transmit the most important data – Implicit adaptation to the network speed – Rendering quality constantly improves 0.1 s (at 1Mbps) 10 s 40 s

16. 16 Adaptive streaming Restricted number of pending requests – Potential LOD loading requests come with an importance value – Requests with highest importance are transmitted to server – Others are discarded, need to update importance for next selection – At reception, database is updated and new request is transmitted Server File server Network Client Adaptive streaming Adaptive rendering Rendering system Partial database Complete database Importance New LOD Request Reply (data)

17. 17 Client database Incomplete tree of blocks – Data on all leaves entire surface can be rendered – Asynchronous update operations – Tree update requests selected the same way as loading requests Server Partial database Adaptive streaming Tree update Adaptive rendering Refine Split & merge Rendering Requests Adaptive selection

18. At each frame, we: – Hierarchically cull blocks that are not visible – Compute importance for each block to select LOD to be rendered – Trigger requests when unavailable LODs are selected – Select most important requests from triggered ones – Render visible leaf blocks at selected LOD using sample masks Adaptive rendering 18

19. LOD0 19 User selects desired rendering speed – Adaptive quality factor in importance formula – Guides selection of LODs and triggering of operations Adaptive selection LOD1 Importance LOD0 LOD-1 LOD1 usedLOD0 requested, LOD1 used SplitMergeLOD0 usedChildren used 50 frames/s 100 frames/s 150 frames/s

20. Results Tested with 3D terrain rendering application – CPU usage: about 5-10% of each frame time – No stutters due to database updates (asynchronous) – Rendering quality improves as fast as network allows – Stable frame rate as long as enough data are loaded Scales well from high to low performance clients (set-top box) 20 Puget Sound database (Washington state, USA)

21. 21 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

22. 22 Hardware 3D rendering We render 3D geometry with triangles – Conversion from elevation in refine operation – Adapted measure of importance – 3D culling with bounding boxes Geometry caching in graphics hardware – Well suited for discrete LODs – Saves memory transfers (up to +40% rendering speed) Sample masks as triangle strips – Applied directly in hardware – Need to solve cracks on block edges

23. 23 Fixing geometry cracks Additional triangle strip masks on block edges – Block with higher definition adapts – No new samples required – Neighbor knowledge: one per edge – No adaptation needed when more than one There are unsolvable cases – Split and merge constraints

24. 24 Texture maps Color maps rendered using textures – 1 LOD = 1 mipmap – Hardware caching and selection Distinct but linked geometry and texture trees – Specific measures of importance – Single texture coordinates mask for all geometry blocks – Only one texture per geometry block: split and merge constraints Data filtering for down-sampling – Improved quality for low definition LODs – Progressive Texture Map [Marvie03]

25. 25 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

26. Modeling a planet Datum to support ellipsoid reference shape Sphere projected onto a bounding cube – Produces square maps – Saves most redundant data compared to classical solution (25% global) – Avoids visual stretching on poles 26 Google Earth (global, cylindrical) Our solution (cube, gnomonic)

27. 27 Projection adjustment Base: gnomonic 2D map projection – Fast reverse projection (normalization) – 75% less surface on corners than center New adjusted sampling – Planet surface instead of plane of projection – Steps = independent angles of rotation – Two tangent values computed per sample – 33% less surface on corners than center Plane of projection:

28. Crack fixing on edges of the cube – Faces specifically numbered and oriented – Implicit and transparent management Runtime adaptive clipping planes – Good rendering precision, from any distance – Culls hidden parts of the planet (behind the horizon) – No additional comparison 28 Solving specific planet issues 0 12 3 4 5

29. Results Tested on GeForce 9800 GTX+, all features turned on – 140 FPS when asking for 2 million triangles 29 Earth database from NASA

30. 30 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

31. Huge input, huge output – Memory constraints for loading input files – Linear writing to output files preferred First step: re-projection of a planetary map – Project specific points of output window to find input window – Recursive output map subdivision – Load input window when fits in memory and re-project samples Preprocessing 31 1 of 6 output gnomonic maps 1 global input cylindric map

32. Preprocessing Second step: generation of a server database – Direct input window computation – Top-down subdivision into complete tree of blocks – Load input for any sub-tree when fits in memory – Bottom-up construction of block LODs and linear file writing 32 Input mapTree of blocks

33. Results Support for input and output maps of arbitrary size – Projection on a cube: -25% database size – LOD compression with Zlib: up to -75% database size 33 Puget Sound 1.25 GB 740 MB 2mn SRTM 174 GB TrueMarble 41.7 GB 9h 1h30 5h 35mn 14.9 GB 6.8 GB 31 GB 126 GB

34. 34 Plan Introduction Related Work Contributions – Data Structure – Adaptive streaming and rendering – 3D terrain rendering features – Planetary terrains – Preprocessing Conclusion

35. Generic adaptive solution – Sample maps with any size and type – Adapts to network and rendering speeds – Single data structure, generic methods – No redundant data, fast handling Application to 3D terrain rendering – Optimizations for 3D graphics hardware – Most rendering artifacts avoided – Uniformly sampled planet surface – Improved rendering precision Offline generation of huge databases 35

36. Future work Enhance current work – Validate the solution on handheld devices – Use GPU shaders for fast and flexible 3D conversion – Further improve precision with local coordinate systems – Find more specific factors for measure of importance Add new features – Generation of procedural details – Fix for popping artifacts – Advanced LOD compression – Self-shadowing of the terrain – Alternate rendering system with projective grid 36

37. 37 Publications Adaptive Streaming and Rendering of Large Terrains: A Generic Solution. Raphaël Lerbour, Jean-Eudes Marvie, Pascal Gautron. In WSCG 2009. Adaptive Real-time Rendering of Planetary Terrains. Raphaël Lerbour, Jean-Eudes Marvie, Pascal Gautron. Accepted, to be presented at WSCG 2010.