CS/CMPE 524 – High- Performance Computing Outline

CS High-Performance Computing (Au ) - Asim LUMS2 Description (1) Introduction to high-performance computing architectures and programming environments  Performance enhancements for single-processor computers  Data locality, cache optimizations, data dependences, loop optimizations  Parallel computer architectures  Parallel computing performance metrics  Message-passing programming using MPI  Shared-memory programming using OpenMP and P-threads  Dense matrix algorithms  Sorting algorithms

CS High-Performance Computing (Au ) - Asim LUMS3 Description (2) Emphasis on code optimizations and programming techniques  This is not a hardware design course Programming in C/C++ only Programming under Unix/Linux environment

CS High-Performance Computing (Au ) - Asim LUMS4 Goals Mastery of fundamental performance issues for parallel and serial computing Familiarity with shared-memory and message passing architectures and models for parallel program development Hands-on experience in designing and implementing efficient algorithms for high-performance computers Familiarity with the current state-of-art in parallel programming environments, portable software libraries, and program development

CS High-Performance Computing (Au ) - Asim LUMS5 Before Taking This Course… You should be comfortable with… Data structures and algorithms  CS213 is a prerequisite  You should be comfortable with algorithm descriptions and implementations in a high-level programming language Computer organization  CS223 is a prerequisite  CS423 is NOT a prerequisite Programming in C/C++  Majority of work under Unix/Linux environment Linear algebra  Matrix operations, systems of linear equations

CS High-Performance Computing (Au ) - Asim LUMS6 After Taking this Course… You should be able to … understand the basic architectures of single- and multi- processor computers develop and implement programs for distributed- memory and shared-memory parallel computers evaluate and enhance performance on single- and multi-processor computers develop programs in MPI and OpenMP

CS High-Performance Computing (Au ) - Asim LUMS7 Grading Points distribution Assignments (~ 6)35% Quizzes (~ 5)10% Midterm exam25% Final exam (comprehensive)30%

CS High-Performance Computing (Au ) - Asim LUMS8 Policies (1) Quizzes  Quizzes can be announced 1 to 2 days in advance or unannounced Sharing  No copying is allowed for assignments. Discussions are encouraged; however, you must submit your own work  Violators can face mark reduction and/or reported to Disciplinary Committee for action Plagiarism  Do NOT pass someone else’s work as yours! Write in your words and cite the reference. This applies to code as well.  Violators can face mark reduction and/or reported to Disciplinary Committee for action

CS High-Performance Computing (Au ) - Asim LUMS9 Policies (2) Submission policy  Submissions are due at the day and time specified  Late penalties: 1 day = 10%; 2 day late = 20%; not accepted after 2 days  An extension will be granted only if there is a need and when requested several days in advance. Classroom behavior  Maintain classroom sanctity by remaining quiet and attentive  If you have a need to talk and gossip, please leave the classroom so as not to disturb others  Dozing is allowed provided you do not snore loud

CS High-Performance Computing (Au ) - Asim LUMS10 Summarized Course Contents Introduction to single- and multi-processor computing models and architectures Cache tuning and characterization Data locality, data dependences, loop optimization Parallel computer performance metrics, interconnection networks Message-passage programming in MPI Shared-memory programming in OpenMP and P- Threads Parallel dense matrix and sorting algorithms

CS High-Performance Computing (Au ) - Asim LUMS11 Course Material Text  Introduction to Parallel Computing, A. Grama, A. Gupta, G. Karypis, and V. Kumar, 2 nd edition, Addison Wesley, Reference texts  Software Optimization for High Performance Computing, K.R. Wadleigh and I.L. Crawford, Prentice Hall,  The Sourcebook of Parallel Computing, W. Gropp et al., Mrogan Kaufmann,  Using MPI – Portable Parallel Programming with the Message-Passing Interface, 2nd Ed., W. Gropp, E. Lusk, A. Skjellum, MIT Press,  Parallel Programming in OpenMP, R. Chandra et al., Academic Press, 2001.

CS High-Performance Computing (Au ) - Asim LUMS12 Course Web Site For announcements, lecture slides, handouts, assignments, solutions, web resources: The resource page has links to information available on the Web. It is basically a meta-list for finding further information.

CS High-Performance Computing (Au ) - Asim LUMS13 Other Stuff How to contact me?  Office hours: to TR (office: 429)   Please see me during my office hours only. If you need to see me at another time, send me an for an appointment. Philosophy  Knowledge cannot be taught; it is learned.  Be excited. That is the best way to learn. I cannot teach everything in class. Develop an inquisitive mind, ask questions, and go beyond what is required.  I don’t believe in strict grading. But… there has to be a way of rewarding performance.

CS High-Performance Computing (Au ) - Asim LUMS14 Reference Books in LUMS Library High Performance Cluster Computing: Programming and Applications, Volume 2, Rajkumar Buyya (Editor), Parallel Programming Techniques and Applications Using Networked Workstations and Parallel Computers, B. Wilkinson and M. Allen, 1999, 5.3W686P. Introduction to Parallel Processing, Sasikumar et al., Prentice Hall of India, Parallel Computer Architecture: A Hardware/Software Approach, D.E. Culler et al., Computer Organization, Hamacher et al., Computer Organization and Architectural Design for Performance, W. Stalling, Using MPI-2: Advanced Features of the Message Passing Interface, Gropp et al., High Performance Linux Clusters: With OSCAR, Rocks, OpenMosix, and MPI, J.D. Sloan, Sebastopol, How to Build a Beowulf, T.L. Sterling et al., Beowulf Cluster Computing With Linux/Windows, T.L. Sterling (ed.), 2002, 5.43 B481. Parallel I/O for High Performance Computing, J.M. May, 2001.