2. High Resolution Displays at PPPL Mike Miller mmiller@pppl.gov mmiller@eden.rutgers.edu

3. Topics Single Machine Applications –PowerPoint Presentations –PSpice / ExpressPCBPSpiceExpressPCB Stepper Motor Control Circuit –AVS ExpressAVS Express Limitations Parallel Visualization –OpenGL concepts –Systems currently in use: (Custom solutions) WireGL Chromium –How to execute –Performance

4. Single Machine Applications Hardware/Software Required –Pentium 4 1.5GHz –1 GB Ram –40 GB Hard Disk –3 * Matrox G200MMS Quad $699 each –WinNT Required Matrox Virtual Desktop Hack Limitations –PCI Bus 33 MHz * 32bit = 132 MB/sec Total for all devices –Image Size 4096 x 2304 = 9 megapixels 18 MB/image @ 16bit –65,536 colors –7.33fps max 27 MB/image @ 24bit –16,777,216 colors (standard) –4.89fps max

5. Implementing Parallel Visualization Hardware –Dual P4 / 1.5GHz –Myrinet Networking 1280 megabits / second 13  s latency– mostly consistent Uses GM, not TCP/IP –100 Mb switch 100 megabits/second 30  s latency– varies by size –NVIDIA Quadro 2 Pro 1024 x 768 @ 60Hz –Total Resolution: 4096x2304 Software –Debian 2.2r4 ‘testing’ –GM 1.5-pre2b –vncwall 1.0 –WireGL 1.2.1 –Chromium (nightly CVS)

6. OpenGL Overview High level graphics API Not focused on low level (pixel operations), but it’s possible The user defines geometric primitives All primitives have properties associated with them Examples: –Primatives Pixels Vertices –Create triangles –Properties Material Lighting http://www.opengl.org/

7. Example: Atlantis Definition glBegin(GL_POLYGON); glVertex3fv(P001); glVertex3fv(P002); glVertex3fv(P003); glNormal3fv(N01); glEnd(GL_POLYGON); … Rendering glPushMatrix(MASK) / glPopMatrix(MASK) glRotatef(…) glLightfv(…) glMaterialfv(…) glEnable(MASK) etc…

8. How do we Distribute the Display? Customized Solution –Used by main campus –Custom Graphics Apps Each application is rewritten to facilitate selective rendering Time Consuming Impossible for closed source software –Virtual Display Driver Buggy, hard to implement in Windows Intercept Solutions –vncwall Virtual Network Computing –WireGL Intercepts OpenGL calls –Chromium WireGL code base, many additions

9. VNC over a Display Wall Standard VNC –Virtual Network Computing –Developed by AT&T –Allows remote execution & display of applications –Similar to: ssh, export DISPLAY= Distributed VNC –Developed by NCSA –Allows the viewer to run on multiple machines / only display certain tiles –Relatively efficient –Does not implement GLX http://www.ncsa.uiuc.edu/TechFocus/Deployment/DBox/

10. WireGL Replaces OpenGL libraries –opengl32.dll (Windows) –libGL.so (Linux / IRIX) wgl helper application –Copies files to /tmp/wglXXXXX –Executes code in above dir. pipeserver –Runs on the “servers” rey=client, paredXX=servers How it works –libwiregl_client.so intercepts an OpenGL call –Encodes using a custom packet –Sends it to the appropriate machine –Pipeserver converts that WireGL code back to an OpenGL call

11. Normal OpenGL vs WireGL Program libGL.so OpenGL Video Card GLX/DRI Monitor Analog / Digital Video Out Program libwiregl_client.so OpenGL Normal ProgramWireGLed Program Myrinet (Using GM) Machine 1Machine 2 Machine 3Machine 4 Machine 5Machine 6 pipeserver libGL.so Video Card Projector OpenGL GLX/DRI Analog Out

12. GLX vs. WireGL GLX –Designed to interface OpenGL into the X protocol –Minimum state tracking –Not designed for high speed operation –Often replaced by DRI for desktop applications WireGL –Designed for efficient state tracking –Especially good with Vertex Arrays Allows OpenGL to retrieve information directly from the application’s memory Thus, the state tracker can fit it in here.

13. Chromium What’s an SPU? Common SPUs –Pack SPU Send commands directly –Tilesort SPU Distributes “Tiles,” (in our case, 12) –Render SPU Direct Output –Print SPU Logging Program libcrutil_client.so OpenGL crserver libGL.so Video Card Projector OpenGL GLX/DRI Analog Out SPU #1SPU #2SPU #n http://sourceforge.net/projects/chromium/

14. Distributed Visualization General –Many graphics commands must be run in a set order –glFinish() might be too much, since we don’t need actual execution –Use separate contexts to avoid state conflicts WireGL –Supported, but not documented Chromium –Creates multiple threads, each aware of the other –Use “Barriers” to maintain order –Barriers only affect graphics rendering, not program execution

15. Access Grids: Extending the Computational Grid Components of an AG Node Digital Video Digital Audio NETWORK Mixer Control Computer NTSC Video RGB Video Analog Audio Video Capture Computer Display Computer Audio Capture Computer Echo Canceller Group-to-group interactions are different from and more complex than individual-to- individual interactions. Large-scale scientific and technical collaborations often involve multiple teams working together. The Access Grid concept complements and extends the concept of the Computational Grid. The Access Grid project aims at exploring and supporting this more complex set of requirements and functions. An Access Grid node involves 3-20 people per site. Access Grid nodes are “designed spaces” that support the high-end audio/video technology needed to provide a compelling and productive user experience. The Access Grid consists of large-format multimedia display, presentation, and interaction software environments; interfaces to grid middleware; and interfaces to remote visualization environments. With these resources, the Access Grid supports large-scale distributed meetings, collaborative teamwork sessions, seminars, lectures, tutorials, and training. Over 70 AG sites (PPPL will be next!)

16. AG: Providing New Capabilities Capabilities will include –high-quality multichannel digital video and audio, –prototypic large-format display –integrated presentation technologies (PowerPoint slides, mpeg movies, shared OpenGL windows), –prototypic recording capabilities –integration with Globus for basic services (directories, security, network resource management), –macroscreen management –integration of local desktops into the Grid –multiple session capability

18. How to execute: VNC Rey (server) –Start a vncserver Similar to an X server with no display vncserver –geometry 4096x2304 – Run vncwall (Java) Automatically launches tileviewer via vncd Pared Cluster (clients) –Run vncd –Let vncwall (on server) automatically launch tileviewer

19. How to execute: WireGL Rey (client) –wgl Pared Cluster (servers) –Start pipeserver Use a repeating script while (1) pipeserver -f end –Wait for WireGL packets

20. How to execute: Chromium Rey (client) –python2 –crappfaker Pared Cluster (servers) –crserver

21. Implementation Issues Projector Alignment –72 degrees of freedom –Manual Alignment –Automated Alignment Software based –Image warping Hardware based –Stepper Motor Project Projector Discrepencies –Brightness / Contrast Lamp Age –Color balance Trapezoidal Projections –Screen curvature

22. Performance Source: http://graphics.stanford.edu/papers/clust_render/clust_render.pdf 400 3 dataset = approximately 1.5 million triangles 102 patches @ 1800 triangles each = 183,600 triangles <5000 triangles– large polygons, large textures

23. References OpenGL –http://www.opengl.org/ WireGL –http://graphics.stanford.edu/software/wiregl/ Chromium –http://sourceforge.net/projects/chromium/ DisplayWall-in-a-Box –http://www.ncsa.uiuc.edu/TechFocus/Deployment/DBox/ Princeton Main Campus Wall –http://www.cs.princeton.edu/omnimedia/