1 Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University

2 Content Background Architecture Applications Maya(with plug-in) PovRay Mental Ray Resource Matching

3 Objectives Illustration of an ICT application encompassing diverse components in applications, portal & middleware Application has potential to benefit digital media industry Complements promotion of Grid Computing in digital media industry

4 Background Project duration: Nov’2004 till Nov 2006 Initiative of Infocomm Development Authority(iDA) Singapore Hewlett Packard(HP) Participants School of Information Systems, Singapore Management University School of Computer Engineering, Nanyang Technological University Institute of High Performance, A*STAR

5 Background(2)

6 Portal Media Grid Architecture iHPC Meta-scheduler ESC Submit job SOAP ESC Platform LSF Meta-scheduler NGO LSF Grid Submit job WWW Browser Maya submit ssh http ssh animator Toolkits - Data collection - Job monitoring - Results preview - Fault-redundancy (Missing job resubmission) - Input data preprocessing/ uncompressing Applications: - Pixie - Povray - Modeler - MentalRay Resource Matchmaker Mental Ray/others ESC

7 Use scenario Data Media Portal Charging model QM MAYA/3DS,etc. Info service Job service other svcs Virtualization/ Middleware Layer data

8 Work Packages WP1 - Point of access for users. Provide real-time collaborative environment & Media Workbench WP2 - Gridification of animation & image rendering applications WP3 - Generic middleware for job execution & management WP3 - Expanding the scope of HP – Quartermaster tool from UDC to Grid for resource virtualization & allocation WP4 - Globus standards. No development undertaken WP5 - Aims to model usage patterns using NGO/GOG data & work on market based resource allocations models/biz models

9 Multiple Interfaces for Job Submission

10 Step 1 – Create Model in Maya

11 Step 2 – Job Submission from Maya to the Grid Each frame is associated with one Grid job.

12 Step 3 – Output Data Collection from Portal The package contains all resultant TIF files.

13 Speedup Sample The time to render a RIB file is about 40 seconds. 351 RIB files need about 4 hours for rendering in a single computer. Clusters hpc-pdpm has 20 CPUs in 10 machines. Cluster melon has 20 CPUs in 10 machines. By using LSF to submit the jobs to hpc-pdpm and melon, now rending 351 RIB files only need about 14 minutes including the time for data stage in/out. The speedup is about 17.

14 Job Submission from Portal to LSF The uploaded package includes all rib files, textures and shaders.

15 Check Status for the Most Recently Submitted Job Progress

16 Portlet of Checking Status for the Rendering Jobs Submitted from Maya and from Portal

17 2. POV-Ray Interactive Rendering Goals Use Grid to render complex photo-realistic objects when a single computer is not sufficient. Deliver the application through the rich media portal directly to the users, without installing additional software Perform on-the fly visualization of the rendering process to give the user more control.

18 Architecture overview Web interface GRID Cluster A Nodes Cluster B Nodes Portal Proxies Rendering is performed in parallel for every web user

19 POV-Ray Interactive Rendering

20 POV-Ray Interactive Rendering

21 POV-Ray Interactive Rendering Rendering in progress

22 POV-Ray Interactive Rendering

23 3. Interactive Shape Modeling App. Goals Use Grid to work with 3D shapes when a single computer computational power is not sufficient. Deliver the application through the rich media portal directly to the users, without installing additional software Methods Java3D web-based interface to receive user commands and visualize the model Real-time communication link between computing nodes and the interface

24 Architecture overview Web interface GRID Cluster A Nodes Cluster B Nodes Portal Web clients Proxies Rendering is performed in parallel for every web user

25 Task distribution 1. Object is split into several parts 2. The parts are distributed among the processing nodes 3. The results are sent back and combined 4. Users are able to visualize the model In their browsers interactively

26 Interactive Modeling App.

27 Interactive Modeling App

28 Interactive Modeling App

29 4. Mental Ray

30 Mental Ray Arch. Melon020 Mentalray client Melon001 Mentalray server Melon002 Mentalray server Melon003 Mentalray server Melon004 Mentalray server

31 Web-Service Enabled Resource Management Understanding User Requirements Formal Representation of Grid Resources

32 Web-Service Enabled Resource Management Discovery and Matching of Grid Resources Problems Countered in Integration Information Service

33 Conclusion Maya Application plug-in and portal submission Achievements Mental Ray submission through portal. Integration from Application to Information management. Integration with LSF

34 The challenge Thanks!