Download presentation
Presentation is loading. Please wait.
Published byAlannah Adams Modified over 8 years ago
1
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Parallel Programming in C with MPI and OpenMP Michael J. Quinn
2
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Chapter 1 Motivation and History
3
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Outline Motivation Motivation Modern scientific method Modern scientific method Evolution of supercomputing Evolution of supercomputing Modern parallel computers Modern parallel computers
4
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Why Faster Computers? Solve compute-intensive problems faster Solve compute-intensive problems faster Make infeasible problems feasible Reduce design time Solve larger problems in same amount of time Solve larger problems in same amount of time Improve answer’s precision Reduce design time Gain competitive advantage Gain competitive advantage
5
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Definitions Parallel computing Parallel computing Using parallel computer to solve single problems faster Parallel computer Parallel computer Multiple-processor system supporting parallel programming Parallel programming Parallel programming Programming in a language that supports concurrency explicitly
6
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Why MPI? MPI = “Message Passing Interface” MPI = “Message Passing Interface” Standard specification for message-passing libraries Standard specification for message-passing libraries Libraries available on virtually all parallel computers Libraries available on virtually all parallel computers Free libraries also available for networks of workstations or commodity clusters Free libraries also available for networks of workstations or commodity clusters
7
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Why OpenMP? OpenMP an application programming interface (API) for shared-memory systems OpenMP an application programming interface (API) for shared-memory systems Supports higher performance parallel programming of symmetrical multiprocessors Supports higher performance parallel programming of symmetrical multiprocessors
8
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Classical Science Nature Observation Theory Physical Experimentation
9
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Modern Scientific Method Nature Observation Theory Physical Experimentation Numerical Simulation
10
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Evolution of Supercomputing World War II World War II Hand-computed artillery tables Need to speed computations ENIAC Cold War Cold War Nuclear weapon design Intelligence gathering Code-breaking
11
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Supercomputer General-purpose computer General-purpose computer Solves individual problems at high speeds, compared with contemporary systems Solves individual problems at high speeds, compared with contemporary systems Typically costs $10 million or more Typically costs $10 million or more Traditionally found in government labs Traditionally found in government labs
12
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Commercial Supercomputing Started in capital-intensive industries Started in capital-intensive industries Petroleum exploration Automobile manufacturing Other companies followed suit Other companies followed suit Pharmaceutical design Consumer products
13
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 50 Years of Speed Increases ENIAC 350 flops Today > 1 trillion flops
14
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. CPUs 1 Million Times Faster Faster clock speeds Faster clock speeds Greater system concurrency Greater system concurrency Multiple functional units Concurrent instruction execution Speculative instruction execution
15
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Systems 1 Billion Times Faster Processors are 1 million times faster Processors are 1 million times faster Combine thousands of processors Combine thousands of processors Parallel computer Parallel computer Multiple processors Supports parallel programming Parallel computing = Using a parallel computer to execute a program faster Parallel computing = Using a parallel computer to execute a program faster
16
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Microprocessor Revolution Micros Minis Mainframes Speed (log scale) Time Supercomputers
17
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Modern Parallel Computers Top 500 List of World’s Supercomputers Top 500 List of World’s Supercomputers http://www.top500.org http://www.top500.orghttp://www.top500.org Cray Supercomputers Inc. Cray Supercomputers Inc. SGI (Silicon Graphics International Corp.) SGI (Silicon Graphics International Corp.) IBM IBM HP ( HP (Hewlett-Packard)
18
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Copy-cat Strategy Microprocessor Microprocessor 1% speed of supercomputer 0.1% cost of supercomputer Parallel computer = 1000 microprocessors Parallel computer = 1000 microprocessors 10 x speed of supercomputer Same cost as supercomputer
19
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Why Didn’t Everybody Buy One? Supercomputer CPUs Supercomputer CPUs Computation rate throughput Inadequate I/O Software Software Inadequate operating systems Inadequate programming environments
20
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Commercial Parallel Systems Relatively costly per processor Relatively costly per processor Primitive programming environments Primitive programming environments Focus on commercial sales Focus on commercial sales Scientists looked for alternative Scientists looked for alternative
21
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Beowulf Concept NASA (Sterling and Becker) NASA (Sterling and Becker) Commodity processors Commodity processors Commodity interconnect Commodity interconnect Linux operating system Linux operating system Message Passing Interface (MPI) library Message Passing Interface (MPI) library High performance/$ for certain applications High performance/$ for certain applications
22
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Advanced Strategic Computing Initiative U.S. nuclear policy changes U.S. nuclear policy changes Moratorium on testing Production of new weapons halted Numerical simulations needed to maintain existing stockpile Numerical simulations needed to maintain existing stockpile Five supercomputers costing up to $100 million each Five supercomputers costing up to $100 million each
23
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. K computer (Fujitsu Cluster) The K Computer (06/2011 world no. 1), which achieved 8.2 quadrillion floating-point operations per second (petaflops)
24
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Summary (1/2) High performance computing High performance computing U.S. government Capital-intensive industries Many companies and research labs Parallel computers Parallel computers Commercial systems Commodity-based systems
25
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Summary (2/2) Power of CPUs keeps growing exponentially Power of CPUs keeps growing exponentially Parallel programming environments changing very slowly Parallel programming environments changing very slowly Two standards have emerged Two standards have emerged MPI library, for processes that do not share memory OpenMP directives, for processes that do share memory
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.