1. High Performance Computing
Course Introduction
Master Degree Program in Computer Science and Networking
Academic Year
Dr. Gabriele Mencagli, PhD
Department of Computer Science
University of Pisa
High Performance Computing, G. Mencagli
12/11/2018

2. Objectives and Contents
Providing a solid framework of concepts and techniques in high-performance computing
Parallelization methodology and models
Parallel architectures
Performance evaluation (analytic cost models)
Support to parallel programming models and software development tools
A methodology for studying current and future systems!
Technology: state-of-the-art and trends
Parallel processors
Multiprocessors
Multicore / manycore / …
High-performance networks
Shared vs distributed memory architectures
Programming models and their runtime support
Studied according to a
Computer Science
Slant!
High Performance Computing, G. Mencagli
12/11/2018

3. Overview of the Discipline and Approach of the Course
High Performance Computing, G. Mencagli
12/11/2018

4. Technologies and Applications
HPC: an enabling technology for advanced industries and users
Technology pull  evolution of application requirements
Large-scale simulations
Big data, real-time analytics
Business, financial trading
Real-time control
Data-intensive and compute-intensive applications
Technology push  parallel machines are “here to stay”
Processor technology: multi-/many-core ( or even 1,000 cores), GPUs, FPGAs
Memory technology: from traditional DRAM to ‘intelligent’ memories
Interconnect technology: high-bandwidth, low-latency networks (e.g., Infiniband)
Computing power: Petascale ( operations per second) – Exascale ( op/sec)
How many processors/cores are expected in the near future?
Additional challenge: minimization of power/energy consumption (Green Computing)
FPGAs stays for Field Programmable Gate Arrays are integrated circuits composed of several hardware resources (mainly memory elements, registers, logic gates) whose behavior is not predefined as in the majority of the circuits. The user writes a specific behavior for the FPGA using a Hardware Description Language that is compiled and translated into a configuration of the FPGA circuit that implements that behavior decided by the user (of course this can be done with some constraints).
High Performance Computing, G. Mencagli
12/11/2018

5. HPC Computing Platforms
(Public) Cloud:
Internet-scale
on-demand
computing
Single server
HPC machine
(multicore inside)
Data Center
(Private Cloud)
High Performance Computing, G. Mencagli
12/11/2018

6. HPC Computing Platforms
Data Center
(Private Cloud)
Single server
HPC machine
(multicore inside)
(Public) Cloud:
Internet-scale
on-demand
computing
HPC machines
as basic
building blocks
Globally,
very large, distributed
HPC systems
High Performance Computing, G. Mencagli
12/11/2018

7. Top500 List
Sunway is an interconnection technology (as Infiniband) and a manufacturer of multi-core CPUs. The same for Aries.
High Performance Computing, G. Mencagli
12/11/2018

8. Approach of the Course
Study basic HPC methodologies and technologies thoroughly
Cloud computing is based on this knowledge, and adds some technological issues (job scheduling, dynamic QoS control, elasticity, security)
Cloud: 2nd year
Parallel architectures, at multicore level
Shared memory
Interconnection
Clusters of shared-memory machines: data centers
Parallel programming models
How to build and how to use HPC machines, and what can be expected from them (performance models)
Metholological and conceptual imprint, with application to current and future technologies
Not laboratory
2nd year: laboratory, commercial and open-source programming tools
High Performance Computing, G. Mencagli
12/11/2018

9. The “Hardware-Software Gap” in Parallel Computing
High Performance Computing, G. Mencagli
12/11/2018

10. Parallel Processing Models
Parallelization of sequential computations
Which application design methodology?
Which programming technologies?
How to design and how to dimension a parallel program
How to map a parallel program onto a parallel machine
How to evaluate/predict performance
Uniform ‘hardware-software’ methodology
How to model parallel architectures
How to model parallel programs
How parallel programs are designed for, and executed by, parallel architectures
High Performance Computing, G. Mencagli
12/11/2018

11. Course Organization and Working Approach
High Performance Computing, G. Mencagli
12/11/2018

12. Course Program and Lectures
The program will cover two fundamental parts
Parallel Programming Models
Parallel Architectures
We will study the two parts in an interleaved way
We have three lectures per week for a total of 36 lectures (72 hours)
Slightly more than half of the hours will be used for introducing and explaining theoretical concepts. The others will be used for exercises
Par. Models
Par. Arch.s
Tuesday
11-13 room N1
Wednesday
16-18 room L1
Thursday
14-16 room N1
High Performance Computing, G. Mencagli
12/11/2018

13. Organization Matters Question time
Monday 16:30 – 18:30 in my room (287) at the Department of Computer Science
Homeworks
They are proposed every two weeks. They consist in a set of useful exercises related to the last lectures
Not mandatory but warmly adviced!
Exam modality
written + oral exam
Exams
2 midterms: written part (beginning of November, mid part of December) + oral part (beginning of January)
5 re-sits: January , February, June, July, September
Constraint: you have to do the written part and the oral part in the same re-sit (except some exceptional cases)
High Performance Computing, G. Mencagli
12/11/2018

14. Home Page of the Course
The official page of the course is at the following URL
Downloadable material
Slides of the lectures and homeworks
Exam results and solutions
High Performance Computing, G. Mencagli
12/11/2018

15. Slides are not sufficient for a good preparation!
Reference Book
Pisa University Press
Paper edition
Digital edition
100% faithful coverage of course lectures and exercises
(plus other topics: fully separate from exam program)
We will follow accurately the book step-by-step
Slides are not sufficient for a good preparation!
High Performance Computing, G. Mencagli
12/11/2018

16. Errata Corrige
In the home page of the course a small document is available containing integations and the errata corrige of the book
High Performance Computing, G. Mencagli
12/11/2018

17. Working Approach IMPORTANT
Critical aptitude and synthesis capability must be properly developed
Exam: just repeating parts of the book is only the necessary condition – in no way it is sufficient!
Required ability to solve problems by properly combining various course concepts and techniques
IMPORTANT
Interaction with the teacher is strongly recommended
Questions during the lectures
Presentation and discussion of exercises and problems
Homeworks
Question time (“orario di ricevimento”) (in Italian for Italians, if they wish)
High Performance Computing, G. Mencagli
12/11/2018

18. Background and Prerequisites
I will assume that you know some basic concepts of Computer Architectures from your previous studies
To be more precise a Bachelor degree-level course on Structured Computer Architecture
Firmware level structuring
Assembler level, CPU architecture, I/O, communications, compiling
Memory hierarchies and caching
Process level and its run-time support
In Pisa: the course “Computer Architecture” (Bachelor Degree Program in Computer Science) adopts such an approach
Capability to use concepts in studying real, complex systems and their interrelation with software and programming tools
High Performance Computing, G. Mencagli
12/11/2018

19. Background Test Nature of the test Is it part of the Exam? When?
Completely optional
Is it part of the Exam?
Obsolutely not, it is a test useful for the students to check their degree of preparation about background concepts and prerequisities
When?
Second week of the course, first attempt
Second attempt, later on during the course
Modality
Written test with 15 multiple-choice questions about concepts that you should have studied during your Bachelor Degree program
In case of insuccess?
You are kindly adviced to study the Background part of the text book
High Performance Computing, G. Mencagli
12/11/2018

20. Appendix on Background
High Performance Computing, G. Mencagli
12/11/2018

21. Appendix on Background
The student must be aware of the way in which the Appendix on Prerequisites has to be utilized
No specific lecture will be dedicated to Appendix subjects
Instead, prerequisites will be reminded when needed
In no way such SHORT reminders can replace the full treatment of the prerequisite subjects
The student is strongly invited to use the Appendix in order to
be (to become) able of applying the needed concepts and techniques,
and to fill any possible gap whenever it is necessary
Other reference books on prerequisites (optional)
D.A. Patterson, J.H. Hennessy, “Computer Organization and Design: the Hardware/Software Interface”, Morgan Kaufman Publishers Inc.
A. Tanenbaum, “Structured Computer Organization”, Prentice-Hall
High Performance Computing, G. Mencagli
12/11/2018