Real-Time Geometric and Color Calibration for Multi-Projector Displays Christopher Larson, Aditi Majumder Large-Area High Resolution Displays Motivation Large-Area high resolution displays are useful for things such as visualization, virtual reality, simulation, training, or collaboration applications. Currently no single display exists that can display over a large-area and still maintain a high resolution. The typical resolution for a 60” HDTV for example is around 30 pixels/inch; conversely, even though the IBM T221 – one of the highest resolution displays in the world – displays 204 pixels/inch, the display size is only 22”. A common solution to this problem is to tile multiple displays together to form one large-area high resolution display. This solution can be flexible because any number of displays can be used to cover different areas. Commonly, LCD panels or projectors are tiled to create the display. When using LCDs, the display cannot be easily resized (except by adding or removing physical panels) because the size of the panel itself cannot be changed. Also, because the LCD panels have a bezel around the display area, the overall display is not continuous. Projectors, on the other hand, can be resized and can form a continuous image over a large screen. However, the boundaries of the displays are still visible. In addition, aligning the projectors perfectly can require expensive tools to achieve the best results. Rather than creating a perfectly aligned display, a display consisting of unaligned overlapping projectors can be made. However, this display will have significant geometric and color problems. · · · S ummer U ndergraduate 2 R esearch 0 F ellowship in 0 I nformation 6 T echnology Geometry Because the tiles are overlapping And probably rotated and sheared, the geometry sent to each of the projectors must be corrected. Further compounding the problem, commodity projectors usually have some degree of radial distortions that must also be corrected. Real-Time Correction The calibration techniques currently being used are done offline before presentation. However, by utilizing the graphics card of the computer controlling each projector in the display, images can be corrected quickly enough to for real-time 3D rendering and also for videos. Chromium, an application for intercepting and modifying OpenGL commands and sending the commands over a network, can be used to send the geometry of a 3D rendering to each node in the display. By rendering each frame of a video onto a texture with a program like NCSA Pixel Blaster, Chromium is also able to play videos on tiled displays. The geometry can be corrected linearly while it is being sent to each node by using the “Warped Grid” display method in the Tilesort SPU provided with Chromium. Then, each node can intercept the OpenGL stream when the swap buffer method is called, render the scene to a texture, and apply the other corrections within OpenGL fragment shaders to the texture to achieve real-time results. The nonlinear warping can be done by storing the texture lookup coordinates of each pixel of the framebuffer in a texture and doing a dependent texture lookup using a fragment shader. Color correction can be applied by squaring the framebuffer, multiplying the result with an attenuation map (stored in a texture), and then doing a per channel dependent texture lookup in a 1D color lookup table. Results By copying the framebuffer into a texture each frame using the OpenGL method glCopySubTexture2D and then doing all of the corrections within fragment programs, real-time renderings were achieved. The scene complexity does not affect the post-processing speed of the corrections; thus, very complicated scenes can be rendered without significant differences in rendering speeds between corrected and uncorrected renderings. Conclusion We found that real-time corrections were a possibility when rendering videos and 3D scenes through Chromium using a Chromium SPU to intercept the swap buffer method and using fragment programs to do all of the corrections. Here are some applications of a large high resolution display. On l top is a scientific visualization environment. On the bottom left is a flight simulator. On the top is a traffic command and control unit. On the top right you set the infrastructure for such a display by tiling multiple projectors. Color The luminance of a single projector varies over its entire surface. Furthermore when combining multiple projectors, the overlapping segments of the projectors become much brighter than the surrounding single projector areas of the display. Geometric misalignment: On the left you see that the rough alignment of projectors introduce geometric misalignment. On right you see a visible break in the content across the boundary of a projector which breaks the illusion of a single display. Color Variation: On the left you see the 'hot-spot' effect, i.e. the brightness falls off significantly from the center to fringes of a projector. On the right you see the variation in brightness across different projectors and also the doubly high brightness of the overlap regions How to Solve the Problems In previous systems all of the corrections were calculated manually. The current technique used for many overlapping projection systems is to automate the process by using a camera to take many images of the screen (using different patterns and exposure levels) and then to process all of the data to calculate the geometric warping coefficients and the color transfer functions to generate a perceptually seamless display. A nine projector display arranged in 3x3 array. left: Displaty before calibration, middle: Display after geometric calibration, right: display after geometry and color calibration. Problems with Overlapping Projections