1. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Volume Graphics Technology to Tools David R. Nadeau Principal Scientist San Diego Supercomputer Center University of California, San Diego, USA

2. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Technology & Tools Technology = a useful idea –Polygons, images, volumes, transforms, shading, … Tool = an idea put to use –Visualization, art, story telling, games, … As a technology matures, it becomes a tool –We focus more on its use, than its mechanism –Use really explodes when artists begin to use it

3. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Technology & Tools Written storytelling – dramatically advanced by… –Printing press, movable type, word processing, e-mail, Internet, … Music composition – dramatically advanced by… –Standardized notation, equal-tempered tuning, piano, electric guitar, synthesizer, digital signal processing, … Computer graphics – dramatically advanced by… –Hmmm…?

4. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Topics of the Talk History –What has happened so far? Rendering –What can we do now? –What would we like to do? –What might we be able to do in the future? –How might this affect the way we render volumes? Authoring –How might rendering issues affect how we create volumes? –What might future authoring tools look like? History Rendering Authoring

5. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring A Brief Look at Graphics History… What are some major events? What did they enable? Surface vs. Volume graphics Technology vs. Story telling vs. Games History Rendering Authoring Very

6. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of surface graphics Selected Technology events: 1963“Sketchpad” I.E Sutherland, "Sketchpad: A Man-Machine Graphical Communication System," Ph.D. Thesis, MIT, 1962. 1965 Digital line drawing J.E. Bresenham, “Algorithm for computer control of a digital plotter,” IBM Systems Journal, 4(1), 1965. 1971Gouraud shading H. Gouraud, “Continuous shading of curved surfaces,” IEEE Transactions on Computers, C-20, 1971. 1972 Hidden surface removal M.E. Newell, R.G. Newell, T.L. Sancha, “A Solution to the Hidden Surface Problem,” Proceedings of ACM National Meeting, 1972. 1974Polygon clipping I.E. Sutherland, G.W. Hodgman, “Reentrant Polygon Clipping,” CACM, 17(1), 1974. 1975Phong shading B.T. Phong, “Illumination for Computer Generated Pictures,” CACM, 18(6), 1975. 1976 Scene graphs Texture mapping J.H. Clark, “Hierarchical Geometric Models for Visible Surface Algorithms,” CACM, 19(10), 1976. J.F. Blinn, M.E. Newell, “Texture and Reflection in Computer Generated Images,” CACM, 19(10), 1976. 1977 Shadows Standard lighting model F.C. Crow, “Shadow Algorithms for Computer Graphics,” Computer Graphics, 11(2), 1977. J.F. Blinn, “Models of Light Reflection for Computer Synthesized Pictures,” Computer Graphics, 11(2), 1977. 1978Bump mapping J.F. Blinn, “Simulation of Wrinkled Surfaces,” Computer Graphics, 12(3), 1978. History Rendering Authoring

7. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of surface graphics Selected Technology events: 1980Ray tracing T. Whitted, “An improved illumination model for shaded display,” CACM, 23(6), 1980. 1982 Geometry VLSI AutoCAD J.H. Clark, “The Geometry Engine: A VLSI geometry system for graphics,” SIGGRAPH 82. Autodesk Inc. 1983 Particle systems Alias W.T. Reeves, “Particle Systems – A Technique for Modeling a Class of Fuzzy Objects,” SIGGRAPH 83. Alias Research Inc. 1984 Radiosity Shaders SGI IRIS 1000 Wavefront C.M. Goral, K.E. Torrance, D.P. Greenberg, B. Battaile, “Modeling the Interaction of Light Between Diffuse Surfaces,” SIGGRAPH 84. R.L. Cook, “Shade trees,” SIGGRAPH 84. Silicon Graphics Wavefront Inc. 1986Rendering eq. J.T. Kajiya, “The rendering equation,” SIGGRAPH 86. 1989RenderMan PIXAR 1992OpenGL OpenGL ARB 1993Direct3D Microsoft 1994GLINT 300SX 3Dlabs 1995VRML VRML/Web3D Consortium 1997 Voodoo Riva 128 3dfx nVidia History Rendering Authoring Too many papers to list

8. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of surface graphics Selected Story Telling events: 1982 Tron Wrath of Khan Disney Paramount 1984 Andre & Wally B. The Last Starfighter PIXAR Universal 1985Young Sherlock Holmes Paramount 1986Luxo Jr. PIXAR 1987 Red’s Dream Stanley & Stella PIXAR Symbolics 1988Tin Toy PIXAR 1989 Knickknack The Abyss PIXAR Fox 1991 Beauty and the Beast Terminator 2 Disney Tri-Star 1992 Aladdin Lawnmower Man Disney New Line History Rendering Authoring All images copyright by Disney, Fox, Paramount, New Line, PIXAR, and Tri-Star, as appropriate.

9. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of surface graphics Selected Story Telling events: 1993Jurassic Park Universal 1994 The Mask Reboot True Lies New Line MainFrame Fox 1995Toy Story PIXAR 1996 Dragonheart Twister Universal Warner Bros. 1997 Geri’s Game Lost World Star Wars, Special Edition Titanic PIXAR Universal Lucasfilm Paramount 1998 A Bug’s Life Antz PIXAR DreamWorks 1999 Toy Story 2 Star Wars, Episode 1 PIXAR Lucasfilm 2000Dinosaur Disney History Rendering Authoring All images copyright by Disney, DreamWorks, Paramount, Lucasfilm, MainFrame, PIXAR, and Universal, as appropriate.

10. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of surface graphics Selected Game events: 1991Catacomb 3-D Id Software 1992 Wolfenstein 3-D 7th Guest Id Software Tilobyte Studios 1993 DOOM Myst Id Software Cyan Productions 1995Dark Forces LucasArts 1996 Nintendo 64 Quake Tomb Raider Nintendo Id Software Eidos Interactive 1997 Jedi Knight Quake II Riven LucasArts Id Software Cyan Productions 1998 Half-Life Thief Unreal Sierra Eidos Interactive Epic MegaGames 1999Quake III: Arena Id Software 2000RealMyst Cyan Productions Catacomb 3-D DOOM Quake Unreal Jedi Knight History Rendering Authoring Tomb Raider Half-Life Quake III: Arena All images copyright by Id Software, Cyan Productions, Eidos Interactive, Epic MegaGames, LucasArts, and Sierra, as appropriate.

11. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of surface graphics 1960’s1970’s1980’s1990’s2000’s Technology Story Telling Games History Rendering Authoring

12. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of volume graphics Selected Technology events: 1984 Volume ray casting H. Tuy, L. Tuy, “Direct 2D display of 3D objects,” Computer Graphics and Applications, 4(10), 1984. 1985 3D texture generation K. Perlin, “An image synthesizer,” SIGGRAPH 85. 1987Marching cubes W.E. Lorensen, H.E. Cline, “Marching cubes: A high resolution 3D surface construction algorithm,” SIGGRAPH 87. 1989Hypertexture K. Perlin, E.M. Hoffert, “Hypertexture,” SIGGRAPH 89. 1990Splatting L. Westover, “Footprint evaluation for volume rendering,” SIGGRAPH 90. 1991Volume sculpting T.A. Galyean, J.F. Hughes, “Sculpting: An interactive volumetric modeling technique,” SIGGRAPH 91. 1994 Shear-warp 3D texture mapping P. Lacroute, M. Levoy, “Fast volume rendering using a shear-warp factorization of the viewing transformation,” SIGGRAPH 94. B. Cabral, N. Cam, J. Foran, “Accelerated volume rendering and tomographic reconstruction using texture mapping hardware,” Volume Visualization Symposium, 1994. 1999 VolumePro GeForce 256 Real-Time Visualization, Mitsubishi nVidia 2000 GeForce2 DirectX 8.0 nVidia Microsoft History Rendering Authoring Too many papers to list

13. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of volume graphics Selected Story Telling events: 1997Contact Warner Bros. 1998Sphere Warner Bros. History Rendering Authoring Contact Sphere All images copyright by Warner Bros., San Diego Supercomputer Center, and the American Museum of Natural History, as appropriate.

14. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of volume graphics Selected Game events: 2001? ? History Rendering Authoring

15. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of volume graphics 1960’s1970’s1980’s1990’s2000’s Technology Story Telling Games History Rendering Authoring

16. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Evolution of volume graphics Volume graphics today… …is where surface graphics was 15 years ago –We are at the start of a transition from technology to tool What enabled story telling and games for surface graphics? What might do the same for volume graphics? History Rendering Authoring

17. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 1 st business-level (expensive): –Attention-grabbing event: TRON –Interactive rendering hardware: SGI –Authoring tools: Alias, Wavefront 1982-4 1991-31997 1 st consumer-level (cheap): –Attention-grabbing event: Catacomb3D, DOOM –Interactive rendering hardware: Voodoo Enabling events History Rendering Authoring Surface graphics 1960’s1970’s1980’s1990’s2000’s Technology Story Telling Games

18. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 1 st business-level (expensive): –Attention-grabbing event: ? –Interactive rendering hardware: VolumePro, SGI? –Authoring tools: ? 1999 ? 2000 1 st consumer-level (cheap): –Attention-grabbing event: ? –Interactive rendering hardware: nVidia? Enabling events Volume graphics History Rendering Authoring 1960’s1970’s1980’s1990’s2000’s Technology Story Telling Games

19. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Interactive Volume Rendering… History Rendering Authoring What can we do now? What would we like to do? What might we be able to do in the future? How might this affect the way we render volumes?

20. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What can we do now? Canonical volume visualizations… What can we do interactively? History Rendering Authoring

21. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What can we do now? RTviz VolumePro nVidia GeForce2 Ultra SGI IR2 (1 pipe, 4 RMs) Memory256 MB64 MB Fill rateN/A1 Gpixels/s896 Mpixels/s Texture rateN/A2 Gtexels/s768 Mtexels/s Max stored volume 16 x 256 3 (scalar) 256 3 (RGBα) Max volume256 3 @ 30 fps256 3 @ 15 fps256 3 @ 12 fps But 256 3 is a small volume… History Rendering Authoring Data source: Product literature

22. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution can we see? Eye’s lens focuses light onto retina –Fovea = focus area = center 20  More receptors at the fovea –Mostly “Cones” (color) at fovea –Mostly “Rods” (intensity) in surrounding area Fovea Periphery History Rendering Authoring Data source: Perception, Sekuler & Blake, 2 nd ed, 1990, McGraw-Hill.

23. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution can we see? Measure visual acuity by viewing “gratings” of parallel lines How fine a grating can you see? History Rendering Authoring Gratings Data source: Visual Perception, Spillmann & Werner, 1990, Academic Press. Normal vision: 120 lines/degree in center 20  = 2,400 lines Legally blind: 1/10 th normal vision 12 lines/degree in center 20  = 240 lines

24. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution can we see? A 256 3 volume  legally blind! –Very low resolution Normal vision requires  2,400 3 ! –If it only covers central 20  of visual field History Rendering Authoring 180  visual field requires  21,600 3 ! –120 lines/degree in 180  = 21,600 lines!

25. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution can we show? We’re constrained by screen resolution –1280x1024 on a large monitor –30 pixels/degree at normal distance –¼ normal vision = visually impaired For now… –1024 3 is a minimum for effective screen use Smaller volumes are blurry (1 voxel covers multiple pixels) –But we want much higher volume resolutions… History Rendering Authoring

26. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? Medical imaging (cryosections) –State of the art: 2048 x 2048 images, 1/10mm apart –2048 x 2048 x 18000 for full body (180cm person) –Visible Human Male:2048 x 1216 x 1871 Re-digitization of images:4096 x 2700 x 1871 –Visible Human Female:2048 x 1216 x 5189 –Recent brain only:1789 x 1472 x 1152 History Rendering Authoring Head data from the Visible Human Project, www.nlm.nih.gov/research/visible Brain data from Arthur Toga’s lab, LONI, UCLA, www.loni.ucla.edu

27. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? Medical imaging (CT scan) –State of the art: 2048 x 2048 images, ½mm apart –2048 x 2048 x 3600 for full body (180cm person) –Visible Human Male:512 x 512 x 1871 History Rendering Authoring Head data from the Visible Human Project, www.nlm.nih.gov/research/visible

28. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? Microscopy imaging (fluorescence) –State of the art: 1024 x 1024, 0.15 microns apart –1024 x 1024 x 75 for a 10 micron cell –Cell membranes:768 x 768 x 72 History Rendering Authoring Cell data from Hiro Tsukada, Eric Elenko, and Maria Pinhal, UCSD Cancer Center

29. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? Simulations –Tend to use all available memory on a supercomputer –SDSC IBM SP “Blue Horizon” 1152 processors (144 nodes, 8 cpus/node, +others), 1.7 teraflops 576 Gbytes total memory (4 Gbytes/node) Largest possible volume is 5000 3 –Ocean simulation:2160 x 960 x 30 History Rendering Authoring Ocean data from Detlef Stammer, UCSD, Scripps Institute of Oceanography

30. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? Combine to build volumetric scenes –Composite overlapping volumes –Mosaic volumes to fill space –Combine procedural elements History Rendering Authoring

31. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? History Rendering Authoring 256 3 We’re a long way from what we want… why? Full screen Fovea Full eye It’s a logarithmic scale!

32. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Memory and bandwidth Memory needed grows with cube of volume’s width –64 3 RGBα= 1 Mbyte –128 3 RGBα= 8 Mbytes –256 3 RGBα= 64 Mbytes –512 3 RGBα= 512 Mbytes –1024 3 RGBα= 4096 Mbytes Bandwidth needed also grows with frame rate –10 fps minimum –30 fps better –70 fps best History Rendering Authoring

33. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What will volumes require? Memory10 fps30 fps70 fps 256 3 (legally blind) 64 MB640 MB/s1.9 GB/s4.5 GB/s 1024 3 (full screen) 4 GB40 GB/s120 GB/s (30 Gpixl/s) 180 GB/s 2400 3 (fovea) 55 GB550 GB/s1.6 TB/s3.8 TB/s 21600 3 (full eye) 40 TB400 TB/s120 TB/s280 TB/s 120 GB/s is > 30 times current bandwidths! History Rendering Authoring

34. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Growth of pixel fill rates History Rendering Authoring Data source: Product literature SGIPC cards 199630 Mpixels/s3Dlabs Permedia 1997100 Mpixels/snVidia RIVA 128 1998366 Mpixels/s3dfx Voodoo 3 1999540 Mpixels/sATI Rage Fury MAXX 20001000 Mpixels/snVidia GeForce2 Ultra * Not counting custom hardware or special configurations Fill rates recently growing by x2 every year

35. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego How long to get there? Bandwidth @ 30 fpsYears 256 3 (legally blind) 1.9 GB/s (0.5 Gpixels/s) Last year 1024 3 (full screen) 120 GB/s (30 Gpixels/s) 5 years 2400 3 (fovea) 1600 GB/s (400 Gpixels/s) 8 years 21600 3 (full eye) 120000 GB/s (30,000 Gpixels/s) 14 years History Rendering Authoring But, this is not very realistic…

36. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego How long to get there? About trends… “A frequent criticism of predictions of the future is that they rely on mindless extrapolation of current trends without consideration of forces that may terminate or alter that trend.” –Ray Kurzweil, The Age of Spiritual Machines History Rendering Authoring

37. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego How long to get there? About the computer industry… “If the automobile industry had made as much progress in the past fifty years, a car today would cost a hundredth of a cent and go faster than the speed of light.” –Ray Kurzweil, The Age of Spiritual Machines History Rendering Authoring

38. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego CPU performance growth History Rendering Authoring Data source: SPEC benchmark database, www.spec.org Floating point power grows by x2 every 2 years

39. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego CPU performance growth At this rate, by 2020 transistor insulators will be just a few atoms thick –A possible end to growth using this technology –What technology will arrive next? Meanwhile, back to the present… History Rendering Authoring

40. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Memory speed growth History Rendering Authoring Memory bandwidth only grows by x2 every 5 years Data source: Kingston Technology, www.kingston.com PC processors only

41. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Comparing growth rates History Rendering Authoring Highly unlikely that fill rates can continue to grow faster than processor speed and memory bandwidth

42. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What should we do? Don’t bet on rapid fill rate growth to satisfy our volume rendering needs –If fill rate growth drops to track memory bandwidth growth, full screen volume rendering may arrive in 25 years, not 5 years And full eye in 70 years! History Rendering Authoring Faster hardware is no substitute for a better algorithm –How can we change the way we work? –Lots of ways… but I’ll focus on a few…

43. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Change how we render Object-space rendering traversals are in common use –Scan through all voxels and project towards screen Splatting, shear-warp, 3D texture mapping, point clouds –Memory streaming is possible If the viewpoint is outside the volume O(n 3 ) time & space n = volume width History Rendering Authoring

44. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Change how we render Image-space traversals are more time/space friendly –Scan through all screen pixels and project towards voxels Ray casting, … –Memory streaming not possible Nearly random access O(k r n) time & space n = volume width r = image resolution (width x height) k = additional computation factor (for comparisons) History Rendering Authoring

45. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Is volume ray casting really O(k r n)? –Worst case: each ray takes longest path = n –Normal case: early ray termination reduces it –Rays diverge, skipping voxels, introducing aliasing –For accuracy: supersample –For speed: use volume MIP-mapping Upfront cost to build MIP-map volumes Increases memory needed (but not bandwidth) –Memory is cheap, bandwidth is not Change how we render History Rendering Authoring 3

46. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego To interactively render larger volumes, we need to get on the better curves – such as ray casting Rendering time growth History Rendering Authoring full screenfovealegally blind

47. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What can we do? Changing our rendering algorithm can help –For “large data,” image-space traversals are better than object-space traversals –We have “large data” now History Rendering Authoring Cross-over What about changing our data?

48. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Volume Authoring… History Rendering Authoring How might rendering issues affect how we create volumes? What might future authoring tools look like?

49. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego CPU vs. memory speeds History Rendering Authoring PC processors, excluding RDRAM & DDR SDRAM Main memory access (cache miss) Slower memory speed growth means memory references are getting more costly, in terms of cycles Data source: SPEC benchmark database, www.spec.org

50. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego CPUs per Supercomputer History Rendering Authoring Average of top 100 Average of top 500 Adapt to slow memory by adding CPUs w/own memory CPUs/Supercomputer grows by x2 every 3 years Data source: Top 500 Supercomputers, www.top500.org

51. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Cycles & bandwidth Cycles are more available than memory bandwidth –“Easier” to add CPUs than increase bandwidth –Parallel computing is an obvious industry trend But CPU coordination and data sharing is still bandwidth limited History Rendering Authoring Can we trade computation for memory accesses? –Data compression is clearly needed Texture compression nearly standard in graphics hardware Geometry compression becoming available –Additional techniques to reduce memory access costs Interleaved frame buffers, Z-buffer cache View frustum culling, occlusion culling

52. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring with cycles “Shaders” compute parts of the scene as needed –Already common-place in software rendering “Data amplification” – small number of parameters generates large amount of scene content –Programmable graphics pipelines arriving now –For surface graphics: splines, procedural textures, particles –For volume graphics: voxelization, procedural volumes History Rendering Authoring

53. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring with cycles Shaders aren’t just for shading any more –Procedural content creation Noise and turbulence functions –Voxelization based upon geometry “parameters” Voxelize on demand – don’t prevoxelize Authoring uses a mix of data and shaders –Import data sets (CT, MRI, cryosection, etc.) –Sculpt & 3D paint volumes –Program “shaders” History Rendering Authoring

54. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring with shaders Procedural turbulence creates “clouds” –Defines a color & opacity at each point in space –Animate parameters to evolve the cloud –Voxelize during ray-casting No volumes created Very low memory use History Rendering Authoring

55. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring with shaders Manipulate shapes –Compute effects during ray-casting No additional volumes created For each point in space: –Compute shortest distance to surface –Perturb the distance with turbulence –Map distances to opacity History Rendering Authoring Compute distances Add turbulenceMap to opacityDo at high resolution

56. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring with shaders Constructing shapes –Constructive Solid Geometry (CSG) –Cut using primitive shapes (or anything) –Transform and “cut” during ray-casting No pre-cut volumes created History Rendering Authoring

57. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring with shaders Repeat to build volumetric scenes –Multiple data sets, geometry, shaders, … –Evaluate some or all during ray-casting Rendering is an active part of authoring History Rendering Authoring

58. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring scenes Several existing scene structure metaphors –CSG trees= combine primitives (most CAD apps.) –Composite trees= combine images (Houdini, Shake) –Shade trees= combine shaders (RenderMan) –Scene graphs= lay out data sets (VRML, Java3D) –Expression trees= compute scene (any programming lang.) “Compilation” prevoxelizes some, all, or none of the scene –Tune to minimize error, memory use, rendering time History Rendering Authoring

59. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Visualizing the Orion nebula Fluorescing clouds of hydrogen, helium, oxygen,... –Complex structure with pillars, swirls, ripples, and cavities Eagle NebulaLagoon NebulaOrion Nebula History Rendering Authoring

60. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Build a surface for the ionization front –Derived from visual and infrared data Make it fuzzy –Perturb distance field with turbulence Project a color-corrected Hubble image through it –Jitter to reduce streaking Visualizing the Orion nebula History Rendering Authoring

61. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Visualizing the Orion nebula Add 85 proplyds (protostars), shock fronts, … –Each built in a similar way Fly around in it all History Rendering Authoring

62. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Visualizing the Orion nebula History Rendering Authoring

63. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Visualizing the Orion nebula 2112 shader nodes in the full scene –Surfaces, turbulence, image projection, transforms, … Prevoxelized as 86 volumes –2 Gbytes of multi-resolution data –40 Gbytes if we didn’t voxelized separately Ray-caster composited volumes and stars History Rendering Authoring

64. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Concluding remarks What are some directions to explore? History Rendering Authoring

65. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data directions Volume registration –Align volumes for compositing Volume compression –Make it take less space Volume decimation –Express it well at a smaller size “Multimodal Medical Volume Registration Based on Spherical Markers” “Geometric Processing of Volumetric Objects” “Exploiting Eigenvalues of the Hessian Matrix for Volume Decimation” At WSCG 2001 “Towards Continuous Image Representations”

66. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Rendering directions Rendering from compressed data –Smaller data = less bandwidth needed Large (out-of-core) data rendering –I/O during rendering to get data Parallel rendering –More processors to get more bandwidth Hybrid volume + geometry rendering –Keep shapes in their smallest format “A New Parallel Volume Rendering Algorithm” At WSCG 2001 “Parallel Ray Tracing with 5D Adaptive Subdivision”

67. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring directions Content creation “shaders” –Create using procedures Volume sculpting / 3D painting –Create from scratch in an artist-natural way Volume scene construction –Compose, composite, cut, … Volume scene compilation –Prevoxelize for best memory/CPU use At WSCG 2001 “Hypertexturing Complex Volume Objects”

68. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Tool directions Work with users! –Put technology into user’s hands … make it a tool –Help create that attention-grabbing event ?

69. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Acknowledgements USA National Science Foundation USA Department of Energy San Diego Supercomputer Center –Mike Bailey –Steve Cutchin –Alex DeCastro –Eric Enquist –Jon Genetti –Mike Houston –John Moreland