1. SpiderGL A JavaScript 3D Graphics Library for Next-Generation WWW Marco Di Benedetto, Federico Ponchio, Fabio Ganovelli, Roberto Scopigno Visual Computing Lab – ISTI – CNR

2. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 2 3D Content and the WWW Many attempts have been done so far  ActiveX objects  Proprietary browser extensions or plug-ins Most libraries/plugins are too high level to directly access underlying graphics system  Developers forced to comply to some paradigm (scene graphs)  CG experts often find limitations due to high-level abstraction Standardization needed  WebGL : JavaScript bindings to OpenGL|ES 2.0

3. Outline Introduction to the WebGL Rendering Pipeline The SpiderGL Library Case Study: Mesh Rendering Library Features & Demos Conclusions & Future Work Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 3

4. 4 What is WebGL Actually what will be (standardization in progress) Specification owned by the Khronos Group Supported (and defined) by all major web browsers devs JavaScript bindings to OpenGL|ES 2.0 Almost 1-to-1 mapping, some modifications to meet JS developers’ habits and security issues Enables home computers and mobile devices to natively access 3D content directly from web pages (no external plug-ins)

5. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 5 OpenGL|ES 2.0 Stripped version of OpenGL, focused on Embedded Systems Programmable : NO fixed function pipeline  Immediate mode (glBegin / End)  Use vertex / index buffers  Transform Stage (matrix stacks)  Explicit shaders uniforms  Lighting (lighting model, light sources, materials)  Lighting computation through shaders code  Named vertex attributes (glVertexPointer(…), …)  Generic attributes through glVertexAttribPointer(index, …) Data entirely resides on GL resources (buffers, textures)  Buffer Centric API Restrictions  Texture formats, data types, shading language limitations,...

6. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 6 Framebuffer OpenGL|ES 2.0: Architecture (simplified) 0N-1 … Vertex Attributes Stream Vertex Puller Index Stream Vertex Processor Fragment Processor Primitive Assembler & Rasterizer Primitive Assembler & Rasterizer Output Merger VSFSUNI Code Memory Pixel Operations VBRBIBTEX Data Memory FB Screen 0M-1 Texture Units … Uniforms

7. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 7 OpenGL|ES 2.0: API Structure Context  Capabilities Query  Error Query  Creation / Activation / Destruction not part of specifications (EGL for this) Objects  Resources creation / edit / bind / destruction  Data Set & Get Configurable Stages  Enable / Disable  Constants / Parameters Set & Get Programmable Stages  Accept user-defined programs (shaders) Vertex Pipeline  Geometric primitives draw (activate Vertex Puller) Pixel Pipeline  Framebuffer clear & readback (activate Pixel Operations) Synchronization  Command Buffer flush & wait Pipeline Configuration & Data Execution

8. Motivations: Bring CG to the WWW WebGL as a link between Web and CG developers  But... it’s very low-level! Gaps to be filled:  Speed, of course  Algebraic and Geometric structures & algorithms  Asynchronous data fetch facilities  Assets loading  Ease the use of WebGL SpiderGL aims at filling these gaps Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 8

9. SpiderGL: 3D Graphics for WWW Lightweight LGPL library ( http://spidergl.org )http://spidergl.org  provides typical CG data structures & algorithms Sits on top of WebGL for realtime rendering Goals:  Efficiency asymptotic bounds are not the only concern  Simplicity & Short Learning Time Key for fast application coding  Flexibility Help with common tasks and create a robust middle layer for more complicated designs Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 9

10. Leading desing considerations Using a third-party library should be easy  Do not impose any design choice a priori Avoid over-abstraction  In CG simple and self-contained types works very well Users must be able to reuse as much as possible of their former knowledge on the subject  All the entities have de-facto standardized names and behavior  Experienced users often want fine control  low-level access Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 10

11. SpiderGL Architecture Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 11 GL: rendering Mesh: editable or renderable 3D model Space: geometric s. & a. Async: asinchronous data requests managing UI: user and html interface

12. Case Study: Mesh Rendering How a typical 3D model is represented:  A list of vertices (each vertex is a bundle of data)  A list of vertex indices, specifying how they are connected to form basic geometric primitives (points, segments, triangles) What we need to draw it  POV intrinsic and extrinsic parameters  Some procedure to make light sources and model material interact to form colors on the final image Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 12 Vertex & Index BuffersShaders & Uniforms

13. Meshes in SpiderGL Mesh Structure  Set of named vertex attribute streams (position, normal, color, custom,...) with NO predefined semantic  Set of named array or indexed primitive streams (points list, triangulated surface indices, wireframe segments,...) Mesh twins:  SglMeshJS: an editable data structure  SglMeshGL: a renderable graphics resource Unified interface, conversion functions Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 13

14. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 14 Create a Renderable Mesh /*******************************************************************/ var mesh = new SglMesh(gl); // add a vertex attribute named “position” (type is inferred) mesh.addVertexAttribute("position", 3, nativePositions); // add a NORMALIZED vertex attribute named “color” mesh.addVertexAttribute("color", 3, nativeColors, true); // add connectivity information named “triangles” (type is inferred) mesh.addIndexedPrimitives("triangles", gl.TRIANGLES, nativeIndices); // add “array” primitives named “vertices” mesh.addArrayPrimitives("vertices", gl.POINTS, 0, 3); /*******************************************************************/

15. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 15 Create a Shader Program /*******************************************************************/ // vertex shader source code string var vertSource = ""; vertSource += “uniform mat4 u_mvp; \n"; vertSource += "..."; vertSource += "attribute vec3 a_position; \n"; vertSource += "attribute vec3 a_color; \n"; vertSource += "..."; // fragment shader source code string var fragSource = "..."; // program setup var prog = new SglProgram(gl, [vertSource], [fragSource]); /*******************************************************************/

16. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 16 Stream Mapping The mesh has 2 vertex attribute streams (position & color) The vertex shader has 2 input vertex attributes  A correspondence between the two sets must be established /**********************************************/ … … … vertSource += "attribute vec3 a_position; \n"; vertSource += "attribute vec3 a_color; \n"; … … … /**********************************************/ /**********************************************************/ … … … mesh.addVertexAttribute("position", 3, nativePositions); mesh.addVertexAttribute("color", 3, nativeColors, true); … … … /**********************************************************/

17. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 17 Rendering /*******************************************************************/ //  connect shader attr “name” to mesh attr “string” var streamMapping = { a_position : "position", a_color : "color" }; //  set program uniform “name” to value “value” var uniforms = { u_mvp : getModelViewProjectionMatrix() }; // utility function (full control available through SglMeshRenderer) sglRenderMeshPrimitives(mesh, "triangles", prog, streamMapping, uniforms); /*******************************************************************/

18. Option Parameters The GL module aims at simplifying common WebGL tasks Creation of WebGL objects  Textures, programs, framebuffers,...  Common sequence of several commands SpiderGL offers reasonable but overridable defaults Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 18 /*********************************************************/ var textureOpts = { minFilter : gl.LINEAR_MIPMAP_LINEAR, generateMipmap : true, onload : function () {... } }; // create texture from remote image var tex = new SglTexture2D(gl, "image_url", textureOpts); /*********************************************************/

19. Dealing with WebGL Objects Contstructors: SpiderGL provides easy-to-use configurable functions to create WebGL Objects Wrappers: each native handle can be wrapped by optimized high-level objects Flexibility: experienced users may want direct low-level control:  obj.handle  native WebGL object  obj.synchronize()  update state after low-level usage Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 19

20. Overcoming Mesh Limitations Ex.: OpenGL|ES 2.0 (and so WebGL) does not allow 32 bit vertex indices  Limited to 64K vertices per chunk SpiderGL uses packed-indexed primitive stream to seamlessly allow very large meshes  Mesh is automatically subdivided into sub-meshes Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 20

21. On the Math / Geometry Side Support CG-related linear algebra entities  2, 3, 4-dimensional vectors, 2x2, 3x3, 4x4 matrices, quaternions  Two choices: operate on native JS arrays or boxing objects Standard geometric entities  Planes, spheres, boxes,...  Intersection queries Transformation Stack Editable Mesh (SglMeshJS)  Several (and more coming) algorithms on trimeshes Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 21

22. Assets, Asynchronous Fetch and UI COLLADA, OBJ, JSON (and more coming) importers Every kind of handled document can be loaded with a uniform asynchronous request interface  Images, 3D Models, shaders code,... Dynamic priority queues for multiresolution  Requests can be interrupted, priority can change, transfer callbacks can be installed  Special watcher object for completion of batched transfers GLUT-like interface and Interactors  first-person camera, trackball,... Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 22

23. What can be done with SpiderGL Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 23 I’m tired up here... Let’s see some demos! http://spidergl.org/code.php

24. SpiderGL to Assist Content Creation Multimedia Web repositories are widespread  Images, Video, Audio, Text,...  What about 3D Repositories? WebGL will bring HW Accelerated 3D Graphics to WWW Lots of 3D databases  We need an effective way to deploy content to users MeShade: a content authoring tool  Let’s have a look!  http://spidergl.org/meshade/index.html http://spidergl.org/meshade/index.html Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 24

25. Conclusions SpiderGL as a Geometry processing and Visualization library Do not over-abstract graphics objects, ease the completion of common task, allow low-level access Real-time performances with WebGL with several case studies Future Work:  Continuous improvements, growing set of algorithms  Toward an automated process to make high-end 3D content to be available to handheld devices Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 25

26. Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 26 Thank You! Questions? marco.dibenedetto@isti.cnr.it http://spidergl.org