Presentation on theme: "Low-Level Plumbing for Media Integration Turner Whitted Microsoft Research."— Presentation transcript:
Low-Level Plumbing for Media Integration Turner Whitted Microsoft Research
JTW/MSR Outline Part I: Implementation chronology Part I: Implementation chronology –Then, Now, Then again … –aka Wheel Of Reincarnation Part II: Architectural musing Part II: Architectural musing –Perceptual/content requirements –Data paths, data types –Not about programming models
JTW/MSR Starting points – E&S Frame Buffer NO fixed function units NO fixed function units Code for basic logic Code for basic logic Controller programmable by designer only Controller programmable by designer only Treated as peripheral Treated as peripheral Ref: Kajiya, J.T., Sutherland, I.E., and Cheadle, E.C., "A Random-Access Video Frame Buffer," Proceedings of the Conference on Computer Graphics, Pattern Recognition, and Data Structure, UCLA Extension, Los Angeles, California, May 14-16, 1975
JTW/MSR Starting points – Ikonas RDS3000 Single, 32-bit wide datapath based on 2901 bit- sliced DSP Single, 32-bit wide datapath based on 2901 bit- sliced DSP Programmed in C (Gary’s Ikonas Assemler) Programmed in C (Gary’s Ikonas Assemler) NO fixed function units NO fixed function units Bound [later memory mapped] to single application Bound [later memory mapped] to single application Ref: N. England, A graphics system architecture for interactive application-specific display functions, IEEE CGA, pp , Jan 1986.
JTW/MSR Starting points – Pixar CHAP SIMD processors SIMD processors Loops, conditional execution Loops, conditional execution Focus on parallel programming issues Focus on parallel programming issues Ref: Adam Levinthal and Thomas Porter, “Chap – A SIMD Graphics Processor,” Proceedings of SIGGRAPH 84, (18) 3, July 1984, pp. 77 – 82.
JTW/MSR Structural evolution Texture engine with remnants of line drawing DNA Texture engine with remnants of line drawing DNA
JTW/MSR Performance Ikonas RDS3000 (1980) Ikonas RDS3000 (1980) –20 MB/s processor to memory –20 MIPS equiv. Pixar CHAP (1984) Pixar CHAP (1984) –240 MB/s processor to memory (P-bus) –64 MIPS peak ATI 9700 (2002) ATI 9700 (2002) –20800 MB/s chip to memory –8400 MIPS
JTW/MSR API abstractions (OpenGL, DX) Fixed structure pipeline accessed through API Fixed structure pipeline accessed through API –API tracks hardware through several generations while maintaining consistency Integrated with mainstream computing Integrated with mainstream computing Easy to program Easy to program –Single largest key to commercial success of graphics systems and applications
JTW/MSR Complete the circle FPGA+memory FPGA+memory NO fixed function NO fixed function Programmable in C (Verilog actually) Programmable in C (Verilog actually) Good for simple prototyping Good for simple prototyping –No API –No device drivers –No SDK
JTW/MSR Part II: Going forward … Motivation Motivation –We’ve tacked every imaginable feature onto what was initially a line drawing pipeline –It’s time to start over –The window of opportunity is wide open
JTW/MSR Going forward … with graphics processors Alternatives Alternatives 3D Raster Graphics Real-time photorealism Parallel execution for scientific applications Parallel execution for interactive applications Feed eyeballs with Integrated media
JTW/MSR Integrated media (partial illustration) Sprites with depth Lumigraph Light field Geometry centric Image centric Warping Interpolation Polygon rendering + texture mapping Fixed geometr y View- dependent geometry View- dependent texture Concentric mosaics
JTW/MSR Geometry Image geometry image 257 x 257; 12 bits/channel 3D geometry completely regular sampling Ref: X. Gu, S. Gortler, H. Hoppe, “Geometry images,” ACM Transactions on Graphics 21(3): (2002)
JTW/MSR From first principles: function Display Processor U-VWL* Content Video Text Line drawing Animated 3D shapes *Ultra-Vast Wasteland Still photos Display Device
JTW/MSR 3D text experiment Olynyk, Mitchell, Snyder, MSR Extend IBR/volume rendering to text Extend IBR/volume rendering to text Superior image reconstruction Superior image reconstruction Higher image quality than mip-mapped texture Higher image quality than mip-mapped texture
JTW/MSR Generic physical blocks General purpose front end General purpose front end Fixed function back end Fixed function back end Read Cache Write Cache Mapper Recon- Struction/ Filtering Common Memory Ref: T. Whitted, “Overview of IBR: Hardware and Software Issues,” ICIP 2000.
JTW/MSR From first principles: implementation Don’t count on quantum GPUs soon – stick to CMOS digital logic Don’t count on quantum GPUs soon – stick to CMOS digital logic Count on CAD more than feature size Count on CAD more than feature size Heat is the enemy Heat is the enemy The economy of commodity DRAM is hard to beat The economy of commodity DRAM is hard to beat –But there is huge performance pressure on DRAM Designers are restricted only by a lack of experience Designers are restricted only by a lack of experience CMOS digital circuitry Commodity DRAM Content
JTW/MSR Design challenges Essence of the problem Essence of the problem –We don’t have a function to implement –We must design specifically for unknown methods –Brute force is prohibited Feeds and speeds …What do we know? Feeds and speeds …What do we know? –Regulated by content But we rarely turn content conventions into quantitative measures But we rarely turn content conventions into quantitative measures –Limited by perception Which we don’t fully understand Which we don’t fully understand –Function of representation If we don’t know the representation, we don’t know the flows If we don’t know the representation, we don’t know the flows In my group’s research we limit flow to HW “sanity” and then work backwards In my group’s research we limit flow to HW “sanity” and then work backwards
JTW/MSR Summary Graphics hardware has nearly completed the circuit back to its starting point Graphics hardware has nearly completed the circuit back to its starting point –Flexible, powerful, programmable Media processing requirements extend beyond the classic 3D pipeline Media processing requirements extend beyond the classic 3D pipeline Unusual window of opportunity Unusual window of opportunity –to match architecture with a broader range of applications and content