1. CSC 7600 Lecture 1 : Introduction Spring 2011 HIGH PERFORMANCE COMPUTING: MODELS, METHODS, & MEANS AN INTRODUCTION Prof. Thomas Sterling Department of Computer Science Louisiana State University January 18, 2011

2. CSC 7600 Lecture 1 : Introduction Spring 2011 Aerial & Satellite of Hurricane Katrina 2

3. CSC 7600 Lecture 1 : Introduction Spring 2011 Devastation from Hurricane Katrina 3

4. CSC 7600 Lecture 1 : Introduction Spring 2011 Simulating Katrina

5. CSC 7600 Lecture 1 : Introduction Spring 2011 5 Evolution of HPC 110 3 10 6 10 9 10 12 10 15 KiloOPSMegaOPSGigaOPSTeraOPSPetaOPSOne OPS 1951 Univac 1 1949 Edsac 1976 Cray 1 1982 Cray XMP 1988 Cray YMP 1964 CDC 6600 1996 T3E 1823 Babbage Difference Engine 1991 Intel Delta 1997 ASCI Red 2001 Earth Simulator 2003 Cray X1 1943 Harvard Mark 1 1959 IBM 7094 2006 BlueGene/L 2009 Cray XT5

6. CSC 7600 Lecture 1 : Introduction Spring 2011 New Fastest Computer in the World DEPARTMENT OF COMPUTER SCIENCE @ LOUISIANA STATE UNIVERSITY 6

7. CSC 7600 Lecture 1 : Introduction Spring 2011 2 nd Fastest Computer in the World 7 Jaguar (Cray XT5-HE) Owned by Oak Ridge National Laboratory Breaks Petaflops processing barrier(1.759e+15 flops) Contains 224,162 AMD x86_64 Opteron Six Core 2600 MHz chips

8. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 8

9. CSC 7600 Lecture 1 : Introduction Spring 2011 9 Synergy Drives Supercomputing Evolution Technology –Enables digital technology –Defines balance of capabilities –Establishes relationship of relative costs Architecture –Creates interface between computation and technology –Determines structures of technology-based components –Establishes low-level semantics of operation –Provides low-cost mechanisms Model of Computation –Paradigm by which computation is manifest –Provides governing principles of architecture operation –Implies programming model and languages

10. CSC 7600 Lecture 1 : Introduction Spring 2011 10 Where Does Performance Come From? Device Technology –Logic switching speed and device density –Memory capacity and access time –Communications bandwidth and latency Computer Architecture –Instruction issue rate Execution pipelining Reservation stations Branch prediction Cache management –Parallelism Parallelism – number of operations per cycle per processor –Instruction level parallelism (ILP) –Vector processing Parallelism – number of processors per node Parallelism – number of nodes in a system

11. CSC 7600 Lecture 1 : Introduction Spring 2011 Major Technology Generations (dates approximate) Electromechanical –19 th century through 1 st half of 20 th century Digital electronic with vacuum tubes –1940s Core memory –1950 Transistors –1947 SSI & MSI RTL/DTL/TTL semiconductor –1970 DRAM –1970s CMOS VLSI –1990 Multicore –2006 11

12. CSC 7600 Lecture 1 : Introduction Spring 2011 12 The SIA ITRS Roadmap

13. CSC 7600 Lecture 1 : Introduction Spring 2011 13 Memory mats: ~ 1 Mbit each Row Decoders Primary Sense Amps Secondary sense amps & “page” multiplexing Timing, BIST, Interface Kerf Classical DRAM Density/Chip has dropped below 4X/3yrsAnd 45% of Die is Non-Memory

14. CSC 7600 Lecture 1 : Introduction Spring 2011 14 Peak Logic Clock Rates 3 GHz Classical Moore’s Law 2005 projection was for 5.2 GHz – and we didn’t make it in production. Further, we’re still stuck at 3+GHz in production. 3 GHz

15. CSC 7600 Lecture 1 : Introduction Spring 2011 15 Classes of Architecture for High Performance Computers Parallel Vector Processors (PVP) –NEC Earth Simulator, SX-6 –Cray- 1, 2, XMP, YMP, C90, T90, X1 –Fujitsu 5000 series Massively Parallel Processors (MPP) –Intel Touchstone Delta & Paragon –TMC CM-5 –IBM SP-2 & 3, Blue Gene/Light –Cray T3D, T3E, Red Storm/Strider Distributed Shared Memory (DSM) –SGI Origin –HP Superdome Single Instruction stream Multiple Data stream (SIMD) –Goodyear MPP, MasPar 1 & 2, TMC CM-2 Commodity Clusters –Beowulf-class PC/Linux clusters –Constellations –HP Compaq SC, Linux NetworX MCR

16. CSC 7600 Lecture 1 : Introduction Spring 2011 Top 500 : System Architecture 16

17. CSC 7600 Lecture 1 : Introduction Spring 2011 17 Driving Issues/Trends Multicore –Now: 8, AMD Opterons, Intel Xeon –possibly 100’s –will be million-way parallelism Heterogeneity –GPGPU –Clearspeed –Cell SPE Component I/O Pins –Off chip bandwidth not increasing with demand Limited number of pins Limited bandwidth per pin (pair) –Cache size per core may decline –Shared cache fragmentation System Interconnect –Node bandwidth not increasing proportionally to core demand Power –Mwatts at the high end = millions of $s per year

18. CSC 7600 Lecture 1 : Introduction Spring 2011 18 Multi-Core Motivation for Multi-Core –Exploits improved feature-size and density –Increases functional units per chip (spatial efficiency) –Limits energy consumption per operation –Constrains growth in processor complexity Challenges resulting from multi-core –Relies on effective exploitation of multiple-thread parallelism Need for parallel computing model and parallel programming model –Aggravates memory wall Memory bandwidth –Way to get data out of memory banks –Way to get data into multi-core processor array Memory latency Fragments L3 cache –Pins become strangle point Rate of pin growth projected to slow and flatten Rate of bandwidth per pin (pair) projected to grow slowly –Requires mechanisms for efficient inter-processor coordination Synchronization Mutual exclusion Context switching

19. CSC 7600 Lecture 1 : Introduction Spring 2011 19 Heterogeneous Multicore Architecture Combines different types of processors –Each optimized for a different operational modality Performance > nX better than other n processor types –Synthesis favors superior performance For complex computation exhibiting distinct modalities Conventional co-processors –Graphical processing units (GPU) –Network controllers (NIC) –Efforts underway to apply existing special purpose components to general applications Purpose-designed accelerators –Integrated to significantly speedup some critical aspect of one or more important classes of computation –IBM Cell architecture –ClearSpeed SIMD attached array processor

20. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 20

21. CSC 7600 Lecture 1 : Introduction Spring 2011 21 Definitions: “supercomputer” Supercomputer: A computing system exhibiting high-end performance capabilities and resource capacities within practical constraints of technology, cost, power, and reliability. Thomas Sterling, 2007 Supercomputer: a large very fast mainframe used especially for scientific computations. Merriam-Webster Online Supercomputer : any of a class of extremely powerful computers. The term is commonly applied to the fastest high-performance systems available at any given time. Such computers are used primarily for scientific and engineering work requiring exceedingly high-speed computations. Encyclopedia Britannica Online

22. CSC 7600 Lecture 1 : Introduction Spring 2011 22 Moore’s Law Moore's Law describes a long- term trend in the history of computing hardware, in which the number of transistors that can be placed inexpensively on an integrated circuit has doubled approximately every two years.

23. CSC 7600 Lecture 1 : Introduction Spring 2011 23 Top 500 List

24. CSC 7600 Lecture 1 : Introduction Spring 2011 24 Performance Performance: –A quantifiable measure of rate of doing (computational) work –Multiple such measures of performance Delineated at the level of the basic operation –ops – operations per second –ips – instructions per second –flops – floating operations per second Rate at which a benchmark program takes to execute –A carefully crafted and controlled code used to compare systems –Linpack Rmax (Linpack flops) –gups (billion updates per second) –others Two perspectives on performance –Peak performance Maximum theoretical performance possible for a system –Sustained performance Observed performance for a particular workload and run Varies across workloads and possibly between runs

25. CSC 7600 Lecture 1 : Introduction Spring 2011 25 Scalability The ability to deliver proportionally greater sustained performance through increased system resources Strong Scaling –Fixed size application problem –Application size remains constant with increase in system size Weak Scaling –Variable size application problem –Application size scales proportionally with system size Capability computing –in most pure form: strong scaling –Marketing claims tend toward this class Capacity computing –Throughput computing Includes job-stream workloads –In most simple form: weak scaling Cooperative computing –Interacting and coordinating concurrent processes –Not a widely used term –Also: coordinated computing

26. CSC 7600 Lecture 1 : Introduction Spring 2011 26 Machine Parameters affecting Performance Peak floating point performance Main memory capacity Bi-section bandwidth I/O bandwidth Secondary storage capacity Organization –Class of system –# nodes –# processors per node –Accelerators –Network topology Control strategy –MIMD –Vector, PVP –SIMD –SPMD

27. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 27

28. CSC 7600 Lecture 1 : Introduction Spring 2011 28 A Brief History of Supercomputing Mechanical Computing –Babbage, Hollerith, Aiken Electronic Digital Calculating –Atanasoff, Eckert, Mauchly von Neumann Architecture –Turing, von Neumann, Eckert, Mauchly, Foster, Wilkes Semiconductor Technologies Birth of the Supercomputer –Cray, Watanabe The Golden Age –Batcher, Dennis, S. Chen, Hillis, Dally, Blank, B. Smith Common Era of Killer Micros –Scott, Culler, Sterling/Becker, Goodhue, A. Chen, Tomkins Petaflops –Messina, Sterling, Stevens, P. Smith,

29. CSC 7600 Lecture 1 : Introduction Spring 2011 29 Practical Constraints and Limitations Cost –Deployment –Operational support Power –Energy required to run the computer –Energy for support facilities –Energy for cooling (remove heat from machine) Size –Floor space –Access way for power and signal cabling Reliability –One factor of availability Generality –How good is it across a range of problems Usability –How hard is it to program and manage

30. CSC 7600 Lecture 1 : Introduction Spring 2011 30 Historical Machines Leibniz Stepped Reckoner Babbage Difference Engine Hollerith Tabulator Harvard Mark 1 Un. of Pennsylvania Eniac Cambridge Edsac MIT Whirlwind Cray 1 TMC CM-2 Intel Touchstone Delta Beowulf IBM Blue Gene/L

31. CSC 7600 Lecture 1 : Introduction Spring 2011 31 Golden Age of Parallel Architecture 1975 – 1992 Vector –Cray-1&2, NEC SX, Fujitsu VPP SIMD –Maspar, CM-2 Systolic –Warp Dataflow –Manchester, Sigma, Monsoon Multithreaded –HEP, MTA Actor-based –J-Machine 1976 Cray 1

32. CSC 7600 Lecture 1 : Introduction Spring 2011 32 Dark Ages of Parallel Computing Technology drivers 1992 to present Killer Micro and mass market PCs High density DRAM High cost of fab lines CSP –Message passing Economy of scale S-curve MPP Weak scaling –Gustafson et al Beowulf, NOW Clusters MPI Ethernet, Myrinet Linux

33. CSC 7600 Lecture 1 : Introduction Spring 2011 Supercomputer Points of Transition Automated calculating –17 th century Stored program digital electronic –1948 Vector –1975 SIMD –1980s MPPs –1991 Commodity Clusters –1993/4 Multicore –2006 33

34. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 34

35. CSC 7600 Lecture 1 : Introduction Spring 2011 35 Driving Factors for HPC Technology trends –Multicore components –Heterogeneous structures and accelerators The 4 Horsemen of the Apocalypse (SLOW) –Starvation (sufficient parallelism and load balancing) –Latency (idle time due to round trip delays) –Overhead (critical path support mechanisms) –Waiting for contention (inadequate bandwidth) Reliability –Single point failure modes cannot be tolerated –Reduced feature size and increased component count Power consumption –Just too much! –Dominating practical growth in mission critical domains Changing application workload characteristics –Data (meta-data) intensive for sparse numerics and symbolics Programmability & ease of use –System complexity, scale and dynamics defy optimization by hand

36. CSC 7600 Lecture 1 : Introduction Spring 2011 36 Sources of Performance Degradation (SLOW) Starvation –Not enough work to do due to insufficient parallelism or poor load balancing among distributed resources Latency –Waiting for access to memory or other parts of the system Overhead –Extra work that has to be done to manage program concurrency and parallel resources the real work you want to perform Waiting for Contention –Delays due to fighting over what task gets to use a shared resource next. Network bandwidth is a major constraint.

37. CSC 7600 Lecture 1 : Introduction Spring 2011 37 The Memory Wall Memory Access Time CPU Time Ratio THE WALL

38. CSC 7600 Lecture 1 : Introduction Spring 2011 38 Microprocessors no longer realize the full potential of VLSI technology 52%/year 74%/year 19%/year 30:1 1,000:1 30,000:1

39. CSC 7600 Lecture 1 : Introduction Spring 2011 39 Amdahl’s Law startend TOTO TFTF startend TATA T F /g

40. CSC 7600 Lecture 1 : Introduction Spring 2011 40 Amdahl’s Law with Overhead startend TOTO tFtF startend TATA v + t F /g tFtF tFtF tFtF

41. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 41

42. CSC 7600 Lecture 1 : Introduction Spring 2011 42 Supercomputing System Stack Device technologies –Enabling technologies for logic, memory, & communication –Circuit design Computer architecture –semantics and structures Models of computation –governing principles Operating systems –Manages resources and provides virtual machine Compilers and runtime software –Maps application program to system resources, mechanisms, and semantics Programming –languages, tools, & environments Algorithms –Numerical techniques –Means of exposing parallelism Applications –End user problems, often in sciences and technology

43. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview – Goals & Content Course Administration Summary Materials for Test 43

44. CSC 7600 Lecture 1 : Introduction Spring 2011 44 Addressing the Big Questions How to integrate technology into computing engines? How to push the performance to extremes? –What are the enabling conditions? –What are the inhibiting factors? How to manage supercomputer resources to deliver useful computing capabilities? –What are the hardware mechanisms? –What are the software policies? How do users program such systems? –What languages and in what environments? –What are the semantics and strategies? What grand challenge applications demand these capabilities? What are the computational models and algorithms that can map the innate application properties to the physical medium of the machine?

45. CSC 7600 Lecture 1 : Introduction Spring 2011 45 Goals of the Course A first overview of the entire field of HPC Basic concepts that govern the capability and effectiveness of supercomputers Techniques and methods for applying HPC systems Tools and environments that facilitate effective application of supercomputers Hands-on experience with widely used systems and software Performance measurement methods, benchmarks, and metrics Practical real-world knowledge about the HPC community Access by students outside the HPC mainstream

46. CSC 7600 Lecture 1 : Introduction Spring 2011 Student Objectives Computational Scientist HPC researcher System Administrators Design Engineers 46

47. CSC 7600 Lecture 1 : Introduction Spring 2011 47 Course Overview: Multiple Segments Introduction –An Overview –Parallel Computer Architecture –Commodity Clusters –Benchmarking –Throughput Computing Distributed Memory - MPI –Communicating sequential processes (CSP) –Enabling Technologies - Networks –MPI programming –Performance measurement (2) Shared Memory – OpenMP –Single Node Architecture –Enabling Technologies – Memory, Core Architectures,.. –Parallel thread computing –OpenMP programming –Performance factors and measurement (1) System Software –Operating Systems –Schedulers and Middleware –Parallel file I/O Advanced Techniques ⁻ Visualization ⁻ Parallel Algorithms ⁻ HPC Libraries Conclusions –What’s beyond the scope of this course –What form will the future of HPC take

48. CSC 7600 Lecture 1 : Introduction Spring 2011 Introduction & Throughput Computing 48 *Project walkthroughs will be held during office hours.

49. CSC 7600 Lecture 1 : Introduction Spring 2011 Distributed Memory & MPI 49 *Project walkthroughs will be held during office hours.

50. CSC 7600 Lecture 1 : Introduction Spring 2011 Shared Memory & OpenMP 50 *Project walkthroughs will be held during office hours.

51. CSC 7600 Lecture 1 : Introduction Spring 2011 Advanced Techniques 51 *Project walkthroughs will be held during office hours.

52. CSC 7600 Lecture 1 : Introduction Spring 2011 System Software 52 *Project walkthroughs will be held during office hours.

53. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Demo 1 : Performance Scalability Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 53

54. CSC 7600 Lecture 1 : Introduction Spring 2011 54 Course Website HPC Course Website can be accessed at: http://www.cct.lsu.edu/csc7600 Course Info: –Syllabus –Schedule Contact Information in the (People Section): email, IM, Phone etc. All course announcements will be made via email and Website. Lecture Slides will be made available on the course website (Course Material Section) Videos of Lectures will be made available on the course website (Course Material Section) after every lecture.

55. CSC 7600 Lecture 1 : Introduction Spring 2011 55 Contact Information Prof. Thomas Sterling tron@cct.lsu.edu (225) 578-8982 (CCT Office) Johnston Hall 320, (225) 578-3320 Office Hours: Tu(1:00 - 3:00 PM) & Th(9:00 – 10:00 AM) Course Secretary : Ms. Terrie Bordelon tbordelon@cct.lsu.edu 302 Johnston Hall (225) 578-5979 Teaching Assistant: Daniel Kogler dkogler@eatel.net Office Hours : Johnston 318 Tuesday 1:40 – 3:00 PM Thursday 9:00 – 10:00 AM

56. CSC 7600 Lecture 1 : Introduction Spring 2011 56 Grading Policy Grading Policy for Graduate Students : Midterm – 20 % Final – 30 % Problem Sets – 25 % Quizzes – 5 % Project – 20 % Grading Policy for Under-Graduate Students : Midterm – 30 % Final – 35 % Problem Sets – 30 % Quizzes – 5 %

57. CSC 7600 Lecture 1 : Introduction Spring 2011 57 Assignments There will be adequately portioned assignments during this course. –Assignments should be turned in as PRINTOUTS to the TA the following TUESDAY BEFORE CLASS. –Assignments should be turned in WORD format / PDF format. NO handwritten assignments will be accepted. –Assignments involving programming problems should have source code printed and attached, and all solution relevant materials (e.g. PBS scripts, commands used for performance measurement etc…) must be well documented and attached. –Source code and all relevant files for assignments involving programming assignments needs to be submitted according to the guidelines mentioned in each problem-set and is due the same time as the assignment (late policy for source code submissions is the same as that of assignments).

58. CSC 7600 Lecture 1 : Introduction Spring 2011 Assignments LATE POLICY: –All assignments should be turned in on the due date BEFORE the CLASS. –Assignments turned in on the same day by 5 PM (Central) will incur a penalty of 30% of the assignment grade. –Assignments turned in BEYOND 5PM (Central) of the due date will receive 0 points irrespective of the work quality. IMPORTANT : –Most of the assignments will need to be run on local supercomputing resources that are shared among several users. –Jobs that you submit WILL get stuck in a queue. –“Queue ate my homework” is NOT an acceptable excuse for not turning homework in. –You are strongly encouraged to start working on assignments as and when they are assigned to avoid inevitable queue wait times. 58

59. CSC 7600 Lecture 1 : Introduction Spring 2011 Graduate Student Projects Term projects are required for Graduate Students Sample Topics –Parallel Image Processing –Application performance measurement –Advanced visualization techniques –Parallel Programming LATE POLICY: –Abstracts turned in later than the assigned date will incur an overall project penalty of 5% –Walkthroughs done later than the assigned date will incur a overall project penalty of 15% –Projects turned in later than the assigned date will NOT be considered for grading and will have an automatic score of 0. 59

60. CSC 7600 Lecture 1 : Introduction Spring 2011 Graduate Student Project Topics Application Scaling : detailed analysis & performance profiling of application(s) based on parameters such as number of processors, application performance bottlenecks, etc.. Application Development : design and develop new parallel applications with simple performance profiling analysis. Architecture Comparative Studies: alternative networks, processors, accelerators 60

61. CSC 7600 Lecture 1 : Introduction Spring 2011 61 Reference Material No Required Textbook Lecture notes (slides), required reading lists (URLs) provided at the end of lectures, some additional notes (on web site), and assignments would be primary sources of material for exams. Students are strongly encouraged to pursue additional reading material available on the internet (and as part of projects).

62. CSC 7600 Lecture 1 : Introduction Spring 2011 62 DEMO: Computing Resources Overview presented by Adam Yates

63. CSC 7600 Lecture 1 : Introduction Spring 2011 63 Computing Resources Arete [ arete.cct.lsu.edu ] 64 compute nodes x 8 cores Quad-core AMD Opteron Processor @ 2.4 Ghz 8 GB RAM per Node 24TB of shared storage 1GB ethernet network interface 10GB Infiniband interconnect

64. CSC 7600 Lecture 1 : Introduction Spring 2011 64 Plagiarism The LSU Code of Student Conduct defines plagiarism in Section 5.1.16: –"Plagiarism is defined as the unacknowledged inclusion of someone else's words, structure, ideas, or data. When a student submits work as his/her own that includes the words, structure, ideas, or data of others, the source of this information must be acknowledged through complete, accurate, and specific references, and, if verbatim statements are included, through quotation marks as well. Failure to identify any source (including interviews, surveys, etc.), published in any medium (including on the internet) or unpublished, from which words, structure, ideas, or data have been taken, constitutes plagiarism;“ Plagiarism will not be tolerated and will be dealt with in accordance with and as outlined by the LSU Code of Student Conduct : http://appl003.lsu.edu/slas/dos.nsf/$Content/Code+of+Conduct? OpenDocument http://appl003.lsu.edu/slas/dos.nsf/$Content/Code+of+Conduct? OpenDocument

65. CSC 7600 Lecture 1 : Introduction Spring 2011 Topics HPC Applications Supercomputing : An Enabler Architecture, Technologies, Programming Models Performance oriented theme Demo 1 : Performance Scalability Brief History of HPC Sources of Performance Degradation Supercomputer System Stack Course Overview - Goals & Content Course Administration Summary Materials for Test 65

66. CSC 7600 Lecture 1 : Introduction Spring 2011 Summary Materials for Test Defining Supercomputer – slide 21 Performance Issues in HPC – slide 24 Scalability – slide 25 Machine parameters affecting performance – slide 26 Driving factors for HPC – slide 35 Sources of performance degradation – slide 36 Supercomputing system stack – slide 42 66

67. CSC 7600 Lecture 1 : Introduction Spring 2011

68. CSC 7600 Lecture 1 : Introduction Spring 2011 ENIAC (Electronic Numerical Integrator and Computer ) Eckert and Mauchly, 1946. Vacuum tubes. Numerical solutions to problems in fields such as atomic energy and ballistic trajectories. 68

69. CSC 7600 Lecture 1 : Introduction Spring 2011 EDSAC (Electronic Delay Storage Automatic Calculator) Maurice Wilkes, 1949. Mercury delay lines for memory and vacuum tubes for logic. Used one of the first assemblers called Initial Orders. Calculation of prime numbers, solutions of algebraic equations, etc. 69

70. CSC 7600 Lecture 1 : Introduction Spring 2011 MIT Whirlwind Jay Forrester, 1949. Fastest computer. First computer to use magnetic core memory. Displayed real time text and graphics on a large oscilloscope screen. 70

71. CSC 7600 Lecture 1 : Introduction Spring 2011 CRAY-1 Cray Research, 1976. Pipelined vector arithmetic units. Unique C-shape to help increase the signal speeds from one end to the other. 71

72. CSC 7600 Lecture 1 : Introduction Spring 2011 CM-2 Thinking Machines Corporation, 1987. Hypercube architecture with 65,536 processors. SIMD. Performance in the range of GFLOPS. 72

73. CSC 7600 Lecture 1 : Introduction Spring 2011 INTEL Touchstone Delta INTEL, 1990. MIMD hypercube. LINPACK rating of 13.9 GFLOPS. Enough computing power for applications like real-time processing of satellite images and molecular models for AIDS research. 73

74. CSC 7600 Lecture 1 : Introduction Spring 2011 Beowulf Thomas Sterling and Donald Becker, 1994. Cluster formed of one head node and one/more compute nodes. Nodes and network dedicated to the Beowulf. Compute nodes are mass produced commodities. Use open source software including Linux. 74

75. CSC 7600 Lecture 1 : Introduction Spring 2011 Earth Simulator Japan, 1997. Fastest supercomputer from 2002-2004: 35.86 TFLOPS. 640 nodes with eight vector processors and 16 gigabytes of computer memory at each node. 75

76. CSC 7600 Lecture 1 : Introduction Spring 2011 BlueGene/L IBM, 2004. First supercomputer ever to run over 100 TFLOPS sustained on a real world application, namely a three- dimensional molecular dynamics code (ddcMD). 76

77. CSC 7600 Lecture 1 : Introduction Spring 2011 77