1. Creating Grid Resources for Undergraduate Coursework John N. Huffman Brown University Richard Repasky Indiana University Joseph Rinkovsky Indiana University Purdue University

2. The Problem: Rendering Animations typically contain thousands of frames, usually 24-30 frames per second Each frame can be computationally complex, and can take hours to render Datasets can be moderate in size, results are usually small per job (< 10 MB) Each frame can be considered an individual computation, thus it plays well to serialized processing

3. Student Rendering Challenges No dedicated resource Rendering software may require non-HPC friendly resources Students work from a variety of platforms, MAC, Windows, Linux, SGI Students typically are not HPC tech aware

4. Indiana University Render Portal Find a suitably large/powerful compute resource for undergraduate student use Find adequate storage Create an interface that is: – Easy – Intuitive – Expandable – Upgradeable – Powerful

5. Computing Resources Available at Indiana University Quarry – IBM HS21 blade server supercomputer – 112 dual Intel Xeon 5335 quad-core processors – Linux OS Big Red – JS21 blade server supercomputer – 512 dual dual-core PowerPC 970MP – Linux OS AVIDD - distributed cluster computer – 192 Dual Intel P4 – Linux OS Student Technology Center (STC) workstations (Condor Pool) – ~2500 Windows based systems for student labs, general computing

6. STC Workstation Condor Pool 2500+ Systems 3 Year life cycle replacement – 512 MB memory minimum – 1.8-3.4 GHz system speed – 100 Mb – 1000 Mb interconnect – Windows based homogeneous software setup Funded through student fees ONLY

7. IU System Comparisons

8. Storage Future migration to Data Capacitor – Lustre file system – 500 TB – Auto cleans/purges – Mounted on Condor Master node, so no direct access

9. Render Portal Interface

10. Render Portal Overview

11. System Modularity Render packages are sent with each job run – No software is installed on workstations – New packages and updates can be added quickly – Old versions of software can be used

12. Render Portal Interface

13. File Upload and Selection

14. Submit Portlet

15. Job Submit and Monitor

16. Job Monitoring The job is submitted to the Condor pool, using the files created by the scripts. A process is created for each render job, that will monitor the progress and update a SQL database with: – Job information (Render package, scene, user) – Frames finished – Average times – Current status

17. Job Queue

18. Finishing the Job When a job is finished, the monitor process will: – Check if the job completed successfully – Create a preview flash movie of the completed images – Create a zip file of completed images – Send e-mail notification to the user with the status of the job – Notify the administrator if there was a problem

19. Detail Job Information and Preview

20. Condor Usage Statistics

22. Performance Evaluation 300 Frame Blender animation – Dual CPU workstation: 21.5 minutes (107 Hours) – BigRed Super computer: 95 minutes – Condor cluster: 173 minutes 504 Frame Maya animation – Dual CPU workstation 5 minutes (42 Hours) – Condor cluster: 68 minutes

23. Typical Render Jobs

24. Credits Marlon Pierce – IU Yu (Marie) Ma – IU Craig Stewart – IU Margaret Dolinsky and her students at IU Albert William - IUPUI NSF – Grant 0504075