SURGICAL SIMULATIONS: IT’S ALL IN A GAME ! Gaming techniques for medical applications. V. Kotamraju, S. Payandeh, J. Dill Experimental Robotics Laboratory, Simon Fraser University. Abstract ABSTRACT SUMMARY Several computer game development techniques can be adapted for surgical simulation. The choice and implementation of a technique must be based on application requirements. render update control interaction computation and display SURGICAL SIMULATIONS real-time data GAME PROGRAMMING Computer games have come a long way since A.S. Douglas' Tic-Tac-Toe in 1952, evolving into a well-understood set of methods, including recent developments in realism and immersiveness for game scenarios. Surgical Simulation, on the other hand, has barely a decade of technological development. Surgical techniques have evolved from direct hands-on maneuvers to indirect minimal-access procedures like Laparoscopy, involving video cameras to show the surgeon the instruments and operating site. Learning these techniques is difficult, but recent advances in computer technology have allowed development of virtual training environments to help the surgeon-to-be. The common component of virtual reality provides an opportunity to apply game programming ideas to such training environments. This poster outlines key techniques in 3D game applications that can enhance current surgical simulation technology. TEXTURE MAPPING Wraps image on surface model. Provides data or patient-specific view. Adds realism. 3D texture arrays contain original data. Dynamic local 2D arrays contain visible surface data. PARTICLE SYSTEMS Dynamic, time-dependent, unconnected mass points. Particles originate, move and die. Stochastic modeling used to control particles. Velocity based on blood-vessel properties. Particle motion influenced by force-field. Simulation of blood-flow and blood-clot. DESIGN PATTERNS Abstract systems of interaction. Between classes, objects, and communication flow. Use object-oriented programming. Provide model reusability. Programming patterns: Spatial index, factory, singleton. Usability patterns: State, focus, progress. SURGICAL SIMULATION FRAMEWORK Deformable modeling. Collision detection and response. Visual and haptic feedback. NETWORK PROGRAMMING Client-server and multi-player systems. Remote surgical training. High performance computing systems. High-speed networks, Grids. USER INPUT Hardware dependency. Keyboard, mouse. Haptics: 6 DOF, force and torque feedback. Organ-force response to users. ARTIFICIAL INTELLIGENCE Finite State Machines: Analyze surgical motions. Path Planning: Guide trajectory of tool. Fuzzy logic: Measure surgical competence. SHADING Illumination (light and shadows). Adds visual details. Components: Ambient, diffuse, specular. Per-vertex or per-pixel. Phong model: Vertex-normal interpolation for face. For curved surfaces. Light mapping: Light effects on base texture. GEOMETRICAL TECHNIQUES Collision detection. Point-inclusion, ray intersection, convex hull. Sweep and prune: Variants for multiple-object collisions. Triangle reduction: Vertex or edge collapse. Progressive mesh. Selection of LOD. Load-time tuning of hardware. PERFORMANCE TUNING Analysis: Profiling, bottlenecks (time, memory). Uniprocessor: Hierarchical structures. Multiprocessors: Parallel processing. Communication bottleneck. Graphics Processing Unit, Parallel Processing Unit. LEVEL OF DETAIL Scene described at multiple resolutions. Discrete (pre-computed) or continuous (on-the-fly). Fast and realistic rendering. Efficient GPU utilization. ‘Popping’ may occur, unsuitable for single, large models. * Images reproduced from existing publications. References provided as handout. * Visibility determination clipping, culling, occlusion testing. Resolution determination LOD analysis. Transformation and lighting Rasterization THE GRAPHICS PIPELINE data-parallel function-parallel * * * * * * parallels pipeline