Creating Grid Resources for Undergraduate Coursework John N. Huffman Brown University Richard Repasky Indiana University Joseph Rinkovsky Indiana University Purdue University

The Problem: Rendering Animations typically contain thousands of frames, usually frames per second Each frame can be computationally complex, and can take hours to render Datasets can be moderate in size, results are usually small per job (< 10 MB) Each frame can be considered an individual computation, thus it plays well to serialized processing

Student Rendering Challenges No dedicated resource Rendering software may require non-HPC friendly resources Students work from a variety of platforms, MAC, Windows, Linux, SGI Students typically are not HPC tech aware

Indiana University Render Portal Find a suitably large/powerful compute resource for undergraduate student use Find adequate storage Create an interface that is: – Easy – Intuitive – Expandable – Upgradeable – Powerful

Computing Resources Available at Indiana University Quarry – IBM HS21 blade server supercomputer – 112 dual Intel Xeon 5335 quad-core processors – Linux OS Big Red – JS21 blade server supercomputer – 512 dual dual-core PowerPC 970MP – Linux OS AVIDD - distributed cluster computer – 192 Dual Intel P4 – Linux OS Student Technology Center (STC) workstations (Condor Pool) – ~2500 Windows based systems for student labs, general computing

STC Workstation Condor Pool Systems 3 Year life cycle replacement – 512 MB memory minimum – GHz system speed – 100 Mb – 1000 Mb interconnect – Windows based homogeneous software setup Funded through student fees ONLY

IU System Comparisons

Storage Future migration to Data Capacitor – Lustre file system – 500 TB – Auto cleans/purges – Mounted on Condor Master node, so no direct access

Render Portal Interface

Render Portal Overview

System Modularity Render packages are sent with each job run – No software is installed on workstations – New packages and updates can be added quickly – Old versions of software can be used

Render Portal Interface

File Upload and Selection

Submit Portlet

Job Submit and Monitor

Job Monitoring The job is submitted to the Condor pool, using the files created by the scripts. A process is created for each render job, that will monitor the progress and update a SQL database with: – Job information (Render package, scene, user) – Frames finished – Average times – Current status

Job Queue

Finishing the Job When a job is finished, the monitor process will: – Check if the job completed successfully – Create a preview flash movie of the completed images – Create a zip file of completed images – Send notification to the user with the status of the job – Notify the administrator if there was a problem

Detail Job Information and Preview

Condor Usage Statistics

Performance Evaluation 300 Frame Blender animation – Dual CPU workstation: 21.5 minutes (107 Hours) – BigRed Super computer: 95 minutes – Condor cluster: 173 minutes 504 Frame Maya animation – Dual CPU workstation 5 minutes (42 Hours) – Condor cluster: 68 minutes

Typical Render Jobs

Credits Marlon Pierce – IU Yu (Marie) Ma – IU Craig Stewart – IU Margaret Dolinsky and her students at IU Albert William - IUPUI NSF – Grant