CS 524 – High- Performance Computing Outline

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS2 Description (1) Introduction to high-performance computing architectures and programming environments  Performance enhancements for single-processor computers  Data locality, cache optimizations, data dependences, loop optimizations  Parallel computer architectures  Parallel computing performance metrics  Message-passing programming using MPI  Shared-memory programming using OpenMP and P-threads  Dense matrix algorithms  Sparse linear system algorithms

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS3 Description (2) Emphasis on code optimizations and programming issues  This is not a hardware design course Programming in C/C++ only Programming under Unix/Linux environment

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS4 Goals Mastery of fundamental performance issues for parallel and serial computing Familiarity with shared-memory and message passing architectures and models for parallel program development Hands-on experience in designing and implementing efficient algorithms for high-performance computers Familiarity with the current state-of-art in parallel programming environments, portable software libraries, and program development

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS5 Before Taking This Course… You should be comfortable with… Data structures and algorithms  CS213 is a prerequisite  You should be comfortable with (matrix) algorithm descriptions and implementations in a high-level programming language Computer organization  CS223 is a prerequisite  CS423 is NOT a prerequisite Programming in C/C++  Majority of work under Unix/Linux environment Linear algebra  Matrix operations, systems of linear equations

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS6 After Taking this Course… You should be able to … understand the basic architectures of single- and multi- processor computers develop and implement programs for distributed- memory and shared-memory parallel computers evaluate and enhance performance on single- and multi-processor computers develop programs in MPI and OpenMP

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS7 Grading Points distribution Assignments (~ 5)35% Quizzes (~ 5)10% Midterm exam25% Final exam (comprehensive)30%

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS8 Policies (1) Quizzes  Quizzes may be announced 1 to 2 days in advance or they may be unannounced Sharing  No copying is allowed for assignments. Discussions are encouraged; however, you must submit your own work  Violators can face mark reduction and/or reported to Disciplinary Committee for action Plagiarism  Do NOT pass someone else’s work as yours! Write in your words and cite the reference. This applies to code as well.  Violators can face mark reduction and/or reported to Disciplinary Committee for action

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS9 Policies (2) Submission policy  Submissions are due at the day and time specified  Late penalties: 1 day = 10%; 2 day late = 20%; not accepted after 2 days  An extension will be granted only if there is a need and when requested several days in advance.

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS10 Summarized Course Contents Introduction to single- and multi-processor computing models and architectures Cache tuning and characterization Data locality, data dependences, loop optimization Parallel computer performance metrics, interconnection networks Message-passage programming in MPI Shared-memory programming in OpenMP and P- Threads Parallel dense and sparse matrix algorithms

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS11 Course Material Texts  Introduction to Parallel Computing: Design and Analysis of Algorithms, V. Kumar, A. Grama, A. Gupta, G. Karypis, Benjamin/Cummins Publishing Co.,  Using MPI – Portable Parallel Programming with the Message-Passing Interface, 2nd Ed., W. Gropp, E. Lusk, A. Skjellum, MIT Press,  Parallel Programming in OpenMP, R. Chandra et al., Academic Press,  Techniques for Optimizing Applications – High Performance Computing, R.P. Garg and I. Sharapov, Sun Microsystems Press,  Introduction to Parallel Processing, M. Sasikumar, D. Shikhare, P.R. Prakash, Prentice-Hall of India, 2000.

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS12 Course Web Site For announcements, lecture slides, handouts, assignments, quiz solutions, web resources: The resource page has links to information available on the Web. It is basically a meta-list for finding further information.

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS13 Other Stuff How to contact me?  Office hours: 2.30 to 4.00 PM TR (office: 132)   Stop by my office  By appointment Philosophy  Knowledge cannot be taught; it is learned.  Be excited. That is the best way to learn. I cannot teach everything in class. Develop an inquisitive mind, ask questions, and go beyond what is required.  I don’t believe in strict grading. But… there has to be a way of rewarding performance.

CS High-Performance Computing (Wi 2003/2004) - Asim LUMS14 Reference Books in LUMS Library High Performance Cluster Computing: Programming and Applications, Volume 2, Rajkumar Buyya (Editor), Parallel Programming Techniques and Applications Using Networked Workstations and Parallel Computers, B. Wilkinson and M. Allen, 1999, 5.3W686P. Parallel Computer Architecture: A Hardware/Software Approach, D.E. Culler et al., Computer Organization, Hamacher et al., Computer Organization and Architectural Design for Performance, W. Stalling, Using MPI-2: Advanced Features of the Message Passing Interface, Gropp et al., How to Build a Beowulf, T.L. Sterling et al., Beowulf Cluster Computing With Linux/Windows, T.L. Sterling (ed.), 2002, 5.43 B481. Parallel I/O for High Performance Computing, J.M. May, 2001.