1. OU Supercomputing Center for Education & Research Henry Neeman, Director OU Supercomputing Center for Education & Research OU Information Technology University of Oklahoma

2. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20052 People

3. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20053 Things

4. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20054 What is Supercomputing? Supercomputing is the biggest, fastest computing right this minute. Likewise, a supercomputer is one of the biggest, fastest computers right this minute. So, the definition of supercomputing is constantly changing. Rule of Thumb: a supercomputer is typically at least 100 times as powerful as a PC. Jargon: supercomputing is also called High Performance Computing (HPC).

5. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20055 What is Supercomputing About? SizeSpeed

6. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20056 What is Supercomputing About? Size: many problems that are interesting to scientists and engineers can’t fit on a PC – usually because they need more than a few GB of RAM, or more than a few 100 GB of disk. Speed: many problems that are interesting to scientists and engineers would take a very very long time to run on a PC: months or even years. But a problem that would take a month on a PC might take only a few hours on a supercomputer.

7. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20057 What is HPC Used For? Simulation of physical phenomena, such as Weather forecasting Galaxy formation Oil reservoir management Data mining: finding needles of information in a haystack of data, such as Gene sequencing Signal processing Detecting storms that could produce tornados Visualization: turning a vast sea of data into pictures that a scientist can understand Moore, OK Tornadic Storm May 3 1999 [2] [3] [1]

8. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20058 What is OSCER? Multidisciplinary center Division of OU Information Technology Provides: Supercomputing education Supercomputing expertise Supercomputing resources: hardware, storage, software For: Undergrad students Grad students Staff Faculty Their collaborators (including off campus)

9. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 20059 Who is OSCER? Academic Depts Aerospace & Mechanical Engr Biochemistry & Molecular Biology Biological Survey Botany & Microbiology Chemical, Biological & Materials Engr Chemistry & Biochemistry Civil Engr & Environmental Science Computer Science Economics Electrical & Computer Engr Finance History of Science Industrial Engr Geography Geology & Geophysics Library & Information Studies Mathematics Meteorology Petroleum & Geological Engr Physics & Astronomy Radiological Sciences Surgery Zoology More than 150 faculty & staff in 23 depts in Colleges of Arts & Sciences, Business, Engineering, Geosciences and Medicine – with more to come!

10. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200510 Who is OSCER? Organizations Advanced Center for Genome Technology Center for Analysis & Prediction of Storms Center for Aircraft & Systems/Support Infrastructure Cooperative Institute for Mesoscale Meteorological Studies Center for Engineering Optimization OU Information Technology Fears Structural Engineering Laboratory Geosciences Computing Network Great Plains Network Human Technology Interaction Center Institute of Exploration & Development Geosciences Instructional Development Program Laboratory for Robotic Intelligence and Machine Learning Langston University Mathematics Dept Microarray Core Facility National Severe Storms Laboratory NOAA Storm Prediction Center OU Office of the VP for Research Oklahoma Climatological Survey Oklahoma EPSCoR Oklahoma Medical Research Foundation Oklahoma School of Science & Math St. Gregory’s University Physics Dept Sarkeys Energy Center Sasaki Applied Meteorology Research Institute

11. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200511 Center for Analysis & Prediction of Storms: daily real time weather forecasting Oklahoma Center for High Energy Physics: particle physics simulation and data analysis using Grid computing Biggest Consumers

12. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200512 Who Are the Users? Over 225 users so far: over 50 OU faculty over 50 OU staff over 100 students about 20 off campus users … more being added every month. Comparison: National Center for Supercomputing Applications (NCSA), after 20 years of history and hundreds of millions in expenditures, has about 2100 users. * * Unique usernames on cu.ncsa.uiuc.edu and tungsten.ncsa.uiuc.edu

13. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200513 What Does OSCER Do? Teaching Science and engineering faculty from all over America learn supercomputing at OU by playing with a jigsaw puzzle (NCSI @ OU 2004).

14. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200514 What Does OSCER Do? Rounds OU undergrads, grad students, staff and faculty learn how to use supercomputing in their specific research.

15. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200515 Current OSCER Hardware Aspen Systems Pentium4 Xeon 32-bit Linux Cluster 270 Pentium4 Xeon CPUs, 270 GB RAM, 1.08 TFLOPs Aspen Systems Itanium2 cluster 66 Itanium2 CPUs, 132 GB RAM, 264 GFLOPs IBM Regatta p690 Symmetric Multiprocessor 32 POWER4 CPUs, 32 GB RAM, 140.8 GFLOPs IBM FAStT500 FiberChannel-1 Disk Server Qualstar TLS-412300 Tape Library

16. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200516 Coming OSCER Hardware (2005) NEW! Dell Pentium4 Xeon 64-bit Linux Cluster 1024 Pentium4 Xeon CPUs, 2240 GB RAM, 6.55 TFLOPs Aspen Systems Itanium2 cluster 66 Itanium2 CPUs, 132 GB RAM, 264 GFLOPs COMING! 2 x 16-way Opteron Cluster 16 AMD Opteron CPUs, 96 GB RAM, 128 GFLOPs COMING! Condor Pool: 750 student lab PCs COMING! National Lambda Rail Qualstar TLS-412300 Tape Library

17. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200517 Hardware: IBM p690 Regatta 32 POWER4 CPUs (1.1 GHz) 32 GB RAM 218 GB internal disk OS: AIX 5.1 Peak speed: 140.8 GFLOP/s * Programming model: shared memory multithreading (OpenMP) (also supports MPI) * GFLOP/s: billion floating point operations per second sooner.oscer.ou.edu

18. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200518 270 Pentium4 XeonDP CPUs 270 GB RAM ~10,000 GB disk OS: Red Hat Linux Enterprise 3 Peak speed: 1.08 TFLOP/s * Programming model: distributed multiprocessing (MPI) * TFLOP/s: trillion floating point operations per second Hardware: Pentium4 Xeon Cluster boomer.oscer.ou.edu

19. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200519 56 Itanium2 1.0 GHz CPUs 112 GB RAM 5,774 GB disk OS: Red Hat Linux Enterprise 3 Peak speed: 224 GFLOP/s * Programming model: distributed multiprocessing (MPI) * GFLOP/s: billion floating point operations per second Hardware: Itanium2 Cluster schooner.oscer.ou.edu

20. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200520 1,024 Pentium4 Xeon CPUs 2,240 GB RAM 20,000 GB disk Infiniband & Gigabit Ethernet OS: Red Hat Linux Enterp 3 Peak speed: 6.5 TFLOPs * Programming model: distributed multiprocessing (MPI) * TFLOPs: trillion calculations per sec New! Pentium4 Xeon Cluster topdawg.oscer.ou.edu DEBUTED AT #54 WORLDWIDE, #9 AMONG US UNIVERSITIES, #4 EXCLUDING BIG 3 NSF CENTERS www.top500.org

21. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200521 Coming! National Lambda Rail The National Lambda Rail (NLR) is the next generation of high performance networking.

22. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200522 Coming! Condor Pool Condor is a software package that allows number crunching jobs to run on idle desktop PCs. OU IT is deploying a large Condor pool (750 desktop PCs) over the course of the Spring 2005. When deployed, it’ll provide a huge amount of additional computing power – more than is currently available in all of OSCER today. And, the cost is very very low.

23. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200523 What is Condor? Condor is grid computing technology: it steals compute cycles from existing desktop PCs; it runs in background when no one is logged in. Condor is like SETI@home, but better: it’s general purpose and can work for any “loosely coupled” application; it can do all of its I/O over the network, not using the desktop PC’s disk; it can use academic research community’s Grid middleware such as Globus, but it doesn’t have to.

24. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200524 Supercomputing at Night Desktop PCs tend to be very active during the workday. But at night, during most of the year, they’re idle. So they can be used for number crunching.

25. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200525 Condor Condor is like SETI@home: it steals compute cycles from existing desktop PCs; it runs in background when no one is sitting at the desk. Condor is better than SETI@home: it’s general purpose and can work for any loosely coupled application; it can do all of its I/O over the network, not using the desktop PC’s local disk.

26. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200526 How is Condor Helpful? Low cost: either nothing (if you have many Linux PCs), OR very little (if you have many Windows PCs). Repurpose idle time on existing desktop PCs – you’ve already paid for the hardware, and most of the software is FREE! Enable research that involves lots of computing. Share multiple independent Condor pools among various institutions (flocking). It provides a quick Grid computing resource, to get regional campuses used to Grids.

27. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200527 Windows vs. Linux Condor runs best on Unix flavors (e.g., Linux). The Windows version is clipped: it doesn’t support automatic checkpointing or automatic job migration. However, desktop PC users demand a pure Windows experience. So, to use Condor in OU IT student PC labs, we need Windows and Linux at the same time. Solution: VMware

28. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200528 VMware VMware is virtual machine software that allows a host OS and one or more guest OSes. VMware is commercial software, not free. For Condor, OU runs Linux as the host OS and Windows as the guest OS. This way, Condor can run directly on Linux and therefore have the maximal set of features, but desktop users can have a pure Windows experience. The lab PCs are configured to boot directly to the Windows login page, so desktop users never know.

29. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200529 Linux/VMware/Windows/Condor Linux VMwareCondor Windows Desktop Applications Science Applications

30. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200530 But I Don’t Want to Manage Linux! Windows VMware Desktop Applications Condor Science Applications

31. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200531 Condor Setup at OU Lab PCs Condor Users Mgmt Nodes Firewall Device Login Nodes

32. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200532 Current Status at OU Pool of test machines in dorm PC lab Submit/management from Neeman’s desktop PC Rollout to multiple labs during summer Total rollout to 750 PCs by end of 2005

33. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200533 What Does OU Plan to Do w/Condor? Loosely coupled problems: many small, independent jobs High Energy Physics: D-Zero experiment Nanotechnology: Monte Carlo simulation Computational Chemistry: molecular dynamics Aerospace Engineering: parameter space searches... and many others.

34. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200534 NSF CI-TEAM Program The NSF Cyberinfrastructure TEAM program is a brand new program. It is providing grants of up to $250,000 for up to 2 years. One of CI-TEAM’s goals is to expand Cyberinfrastructure – for example, supercomputing – to institutions and people that traditionally haven’t had much access.

35. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200535 Our NSF CI-TEAM Project The University of Oklahoma (OU) is leading an NSF CI-TEAM project, submitted May 27 2005. The focus is on setting up Condor pools across the Great Plains region, and beyond. The kickstart application will be BLAST (bioinformatics), but these Condor pools will be available for any appropriate application. Most of the money in OU’s CI-TEAM proposal will go to institutions other than OU, for VMware licenses and PCs to manage the Condor pools.

36. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200536 CI-TEAM Participants So Far At OU OSCER/IT Arts & Sciences: Botany & Microbiology; Chemistry & Biochemistry; Mathematics; Physics & Astronomy; Zoology Engineering: Aerospace & Mechanical Engineering; Civil Engineering & Environmental Science; Chemical, Biological & Materials Engineering; Computer Science; Electrical & Computer Engineering, Industrial Engineering Medicine: Surgery, Radiological Sciences Other Academic Institutions in Oklahoma: Langston U. (minority serving), Oklahoma Baptist U. (4 year), Oklahoma School of Science & Mathematics (high school), St. Gregory’s U. (4 year), U. Central Oklahoma (Masters-granting) Academic Institutions outside Oklahoma: Contra Costa College of CA (2 year), Kansas State U. (PhD), U. Arkansas Fayetteville (PhD), U. Arkansas Little Rock (PhD), U. Kansas (PhD), U. Nebraska (PhD), U. Northern Iowa (Masters)

37. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200537 An Added Bonus OSCER’s user eligibility policy is that anyone can have access to OSCER’s supercomputers, if they are on a project that has an OU faculty or staff member as the Principal or Co-Principal Investigator. So, as a bonus, everyone at participating institutions who is on the CI-TEAM project – and their students – will get FREE access to OSCER’s supercomputers!

38. OU Supercomputing Center for Education & Research OneNet CIO Meeting Thursday August 11 200538 Join our CI-TEAM! If OU’s CI-TEAM proposal is fully funded, it will create one of largest Condor flocks in the world: almost 4,000 PCs. YOU CAN JOIN US! All you need is to commit some PCs – you create your own Condor pool with OU’s help, and then let the rest of the CI-TEAM institutions use them. CI-TEAM will pay for the VMware licenses, and for small schools also a PC as Condor manager.

39. QUESTIONS? Thank you for your attention.