Download presentation

Presentation is loading. Please wait.

Published byBraydon Barrs Modified over 2 years ago

1
Parallel Algorithms on Networks of Processors Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill) School of Computing, Clemson University pargas@clemson.edu

2
Outline What are parallel algorithms? Why use them? Challenges for parallel algorithm designers Choosing a network Partitioning the data Designing the algorithm Example Recurrences (binary tree) Analysis (Speedup and Efficiency) Summary and Conclusions Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 3

3
Why Parallel Computation? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 4

4
Why Parallel Computation? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 5

5
Why Parallel Computation? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 6

6
Why Parallel Computation? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 7

7
Why Parallel Computation? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 8

8
Why Parallel Computation? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 9

9
New Processors Faster and Cheaper Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 10 January 2011

10
Partition the Data Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 11

11
Organize the Processors Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 12

12
Build a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 13

13
Build a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 14

14
Choosing a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 15

15
Choosing a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 16

16
Choosing a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 17

17
Choosing a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 18

18
Choosing a Network Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 19

19
Are There Really Any Multiprocessing Systems in Use Today? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 20

20
Are There Really Any Multiprocessing Systems in Use Today? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 21 Hamburg June 2011 Top 500

21
Supercomputers NEC/HP Tsubame (Japan) Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 22 1.192 petaflops 1.28 quadrillion floating point operations per sec 73,278 Xeon cores Infiniband grid network

22
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 23 Supercomputers Dawning Nebulae (China) 1.27 petaflops 1.36 quadrillion floating point operations per sec 9280 Intel 6-core Xeon processors = 55,680 cores Infiniband grid network

23
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 24 Supercomputers Cray Jaguar (USA) 1.75 petaflops 1.876 quadrillion floating point operations per sec 224,256 AMD cores 3D torus network

24
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 25 Supercomputers NUDT Tianhe-1A (China) 2.566 petaflops 2.75 quadrillion floating point operations per sec 14336 CPUs Undisclosed proprietary network

25
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 26 Supercomputers Fujitsu K (Japan) K = kei = Japanese for 10 quadrillion 8.162 petaflops 9 quadrillion floating point operations per sec 68,544 8-core SPARC64 processors = 548,352 cores 3-dimensional torus network called Tofu

26
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 27 TOP500 Top 500 Computers in the World

27
Where Does that Leave Us? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 28

28
Where Does that Leave Us? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 29 In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!

29
Where Does that Leave Us? In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 30

30
Where Does that Leave Us? In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 31

31
Where Does that Leave Us? In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 32

32
Challenges for Parallel Algorithm Designers Choosing a network Partitioning the problem Designing the parallel algorithm Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 33

33
So Lets Try It: Choosing a network Partitioning the problem Designing the parallel algorithm Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 34

34
So Lets Try It: Elliptic Partial Diff Eqns Choosing a network Partitioning the problem Designing the parallel algorithm Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 35

35
Elliptic PDEs Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 36 Problems involving second-order elliptic partial differential equations are equilibrium problems. Given a region R bounded by a curve C and that the unknown function z satisfies Laplaces or Poissons equation in R, the objective is to approximate the value of z at any point in R. The method of finite differences is an often used numerical method for solving this problem. The basic strategy is to approximate the differential equation by a difference equation and to solve the difference equation.

36
Designing the Algorithm Why Solve Linear Recurrences? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 37

37
Designing the Algorithm Why Solve Linear Recurrences? The problem: Solving PDEs using the Method of Finite Differences leads to Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 38

38
Designing the Algorithm Why Solve Linear Recurrences? The problem: Solving PDEs using the Method of Finite Differences leads to Solving Block Tridiagonal Systems which leads to Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 39

39
Designing the Algorithm Why Solve Linear Recurrences? The problem: Solving PDEs using the Method of Finite Differences leads to Solving Block Tridiagonal Systems which leads to Solving Tridiagonal Systems which leads to Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 40

40
Designing the Algorithm Why Solve Linear Recurrences? The problem: Solving PDEs using the Method of Finite Differences leads to Solving Block Tridiagonal Systems which leads to Solving Tridiagonal Systems which leads to Solving Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 41

41
Designing the Algorithm Why Solve Linear Recurrences? The problem: Solving PDEs using the Method of Finite Differences leads to Solving Block Tridiagonal Systems which leads to Solving Tridiagonal Systems which leads to Solving Linear Recurrences many many many times Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 42

42
Designing the Algorithm Why Solve Linear Recurrences? The problem: Solving PDEs using the Method of Finite Differences leads to Solving Block Tridiagonal Systems which leads to Solving Tridiagonal Systems which leads to Solving Linear Recurrences many many many times Why Use a Binary Tree? Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 43

43
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 44 Key Idea: Successfully solving the original pde problem depends upon solving recurrences quickly and efficiently.

44
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 45 Consider the following set of n equations: x 0 = a 0 x 1 = a 1 + b 1 x 0 x 2 = a 2 + b 2 x 1... x n-1 = a n-1 + b n-1 x n-2

45
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 46 Consider the following set of n equations: x 0 = a 0 x 1 = a 1 + b 1 x 0 x 2 = a 2 + b 2 x 1... x n-1 = a n-1 + b n-1 x n-2 Can we solve for x i in parallel?

46
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 47 For uniformity: x 0 = a 0 + b 0 x -1 b 0 =0, x -1 =dummy variable x 1 = a 1 + b 1 x 0 x 2 = a 2 + b 2 x 1... x n-1 = a n-1 + b n-1 x n-2

47
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 48 For uniformity: x 0 = a 0 + b 0 x -1 x 1 = a 1 + b 1 x 0 x 2 = a 2 + b 2 x 1... x n-1 = a n-1 + b n-1 x n-2 Observe, if x i = a + b x j x j = a + b x k Then Then x i = (a + ba) +bb x k = a + b x k = a + b x k

48
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 49 Notation change x 0 = a 0 + b 0 x -1 C 0,-1 = (a 0,b 0 ) x 1 = a 1 + b 1 x 0 C 1,0 = (a 1,b 1 ) x 2 = a 2 + b 2 x 1 C 2,1 = (a 2,b 2 )... x n-1 = a n-1 + b n-1 x n-2 C n-1,n-2 = (a n-1,b n-1 ) Observe, if x i = a + b x j x j = a + b x k Then Then x i = (a + ba) +bb x k = a + b x k = a + b x k

49
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 50 Notation change x 0 = a 0 + b 0 x -1 C 0,-1 = (a 0,b 0 ) x 1 = a 1 + b 1 x 0 C 1,0 = (a 1,b 1 ) x 2 = a 2 + b 2 x 1 C 2,1 = (a 2,b 2 )... x n-1 = a n-1 + b n-1 x n-2 C n-1,n-2 = (a n-1,b n-1 ) Observe, if x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb)

50
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 51 To summarize: x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb)

51
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 52 To summarize: x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb) One last observation:

52
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 53 To summarize: x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb) One last observation: If any variable is expressed in terms of the dummy variable x -1 (e.g., x 0 = a 0 + b 0 x -1 ) then that variable is solved.

53
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 54 To summarize: x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb) One last observation: If any variable is expressed in terms of the dummy variable x -1 (e.g., x 0 = a 0 + b 0 x -1 ) then that variable is solved. So C i,-1 = (a,b)

54
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 55 To summarize: x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb) One last observation: If any variable is expressed in terms of the dummy variable x -1 (e.g., x 0 = a 0 + b 0 x -1 ) then that variable is solved. So C i,-1 = (a,b) means that b=0

55
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 56 To summarize: x i = a + b x j C i,j = (a,b) x j = a + b x k C j,k = (a,b) Then Then x i = (a + ba) +bb x k C i,j C j,k = = a + b x k C i,k = (a+ba,bb) = a + b x k C i,k = (a+ba,bb) One last observation: If any variable is expressed in terms of the dummy variable x -1 (e.g., x 0 = a 0 + b 0 x -1 ) then that variable is solved. So C i,-1 = (a,b) means that b=0 and that x i = a + b x -1 = a + 0 x -1 = a

56
C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 57 Linear Recurrences

57
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 58 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1

58
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 59 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1

59
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 60 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1

60
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 61 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1

61
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 62 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1

62
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 63 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1

63
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 64 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1

64
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 65 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1 Solved variable

65
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 66 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1 Solved variables

66
Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 67 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1 How do we solve for the other variables?

67
C 6,5 C 7,-1 Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 68 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1 In the downsweep!

68
C 6,5 C 7,-1 Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 69 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1 (x 7,x -1 )

69
C 6,5 (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 70 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 C 7,-1 (x 7,x -1 )

70
C 6,5 (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 71 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 C 7,3 C 3,-1 x3x3 (x 7,x -1 )

71
C 6,5 (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 72 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,5 C 5,3 C 3,1 C 1,-1 C 7,3 C 3,-1 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 )

72
C 6,5 (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 73 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 C 3,1 C 1,-1 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 C 7,5 C 5,3 x1x1

73
C 6,5 (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 74 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 C 7,5 C 5,3 C 3,1 C 1,-1 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 7,x 5 ) (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 )

74
(x 7,x 5 ) C 6,5 (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 75 C 0,-1 C 1,0 C 2,1 C 3,2 C 4,3 C 5,4 C 7,6 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 ) x6x6 x4x4 x3x3 x1x1

75
(x 7,x 5 ) (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 76 C 0,-1 C 1,0 C 3,2 C 4,3 C 5,4 C 6,5 C 7,6 C 2,1 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 ) x6x6 x4x4 x3x3 x1x1 (x 7,x 6 ) (x 6,x 5 ) (x 5,x 4 ) (x 0,x -1 ) (x 1,x 0 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 )

76
(x 7,x 5 ) (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 77 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 ) x6x6 x4x4 x3x3 x1x1 (x 7,x 6 ) (x 6,x 5 ) (x 5,x 4 ) (x 0,x -1 ) (x 1,x 0 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 7,x 6 ) (x 0,x -1 ) (x 6,x 5 ) (x 5,x 4 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 1,x 0 )

77
(x 7,x 5 ) (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 78 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 ) x6x6 x4x4 x3x3 x1x1 (x 7,x 6 ) (x 6,x 5 ) (x 5,x 4 ) (x 0,x -1 ) (x 1,x 0 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 7,x 6 ) (x 0,x -1 ) (x 6,x 5 ) (x 5,x 4 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 1,x 0 ) Leaves contain solutions!

78
(x 7,x 5 ) (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 79 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 ) x6x6 x4x4 x3x3 x1x1 (x 7,x 6 ) (x 6,x 5 ) (x 5,x 4 ) (x 0,x -1 ) (x 1,x 0 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 7,x 6 ) (x 0,x -1 ) (x 6,x 5 ) (x 5,x 4 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 1,x 0 ) But we can do better!

79
(x 7,x 5 ) (x 7,x -1 ) Linear Recurrences Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 80 x3x3 (x 7,x -1 ) (x 7,x 3 ) (x 3,x -1 ) x5x5 x1x1 (x 5,x 3 ) (x 3,x 1 )(x 1,x -1 ) x6x6 x4x4 x3x3 x1x1 (x 7,x 6 ) (x 6,x 5 ) (x 5,x 4 ) (x 0,x -1 ) (x 1,x 0 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 7,x 6 ) (x 0,x -1 ) (x 6,x 5 ) (x 5,x 4 ) (x 4,x 3 ) (x 3,x 2 ) (x 2,x 1 ) (x 1,x 0 ) Pipelining the solution

80
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 81 Linear Recurrences Pipelining the solution

81
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 82 Linear Recurrences Pipelining the solution

82
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 83 Linear Recurrences Pipelining the solution

83
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 84 Linear Recurrences Pipelining the solution

84
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 85 Linear Recurrences Pipelining the solution

85
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 86 Linear Recurrences Pipelining the solution

86
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 87 Linear Recurrences Pipelining the solution

87
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 88 Linear Recurrences Pipelining the solution

88
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 89 Linear Recurrences Pipelining the solution

89
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 90 Linear Recurrences Pipelining the solution

90
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 91 Linear Recurrences Pipelining the solution

91
Analysis T1 = Time on one processor Tn = Time on n processors S = Speedup = T1/Tn (ideal: S = n) E = Efficiency = S/n (ideal: E = 1) Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 92

92
Speedup and Efficiency Single Pass Assume 1 floating point operation requires 1 time unit T1 = (n1) (1 mult + 1 add) = 2n2 time units n leaves 2n processors T2n = (log 2 n)( 2 mults + 1 add) // upsweep + (log 2 n)(1 mult + 1 add) // downsweep T2n = (log 2 n)( 2 mults + 1 add) // upsweep + (log 2 n)(1 mult + 1 add) // downsweep = 5 log 2 n time units S = Speedup = T1/T2n = (2n2)/(5 log 2 n) E = Efficiency = S/2n = (2n2)/[ (5 log 2 n) (2n) ] Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 93

93
Speedup Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 94 Speedup and Efficiency Single Pass

94
Speedup Efficiency Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 95 Speedup and Efficiency Single Pass

95
With Pipelining. Assume kn equations for large k T1 = (kn1) (1 mult + 1 add) = 2kn2 time units n leaves 2n processors T2n = (log 2 n)( 2 mults + 1 add) // pipefill up + (k – 2 log 2 n) (5) // pipeline on k2log 2 n waves T2n = (log 2 n)( 2 mults + 1 add) // pipefill up + (k – 2 log 2 n) (5) // pipeline on k2log 2 n waves + (log 2 n)( 1 mult + 1 add) // pipedrain down + (log 2 n)( 1 mult + 1 add) // pipedrain down = 5(k – log 2 n) time units S = Speedup = T1/T2n = (2kn2)/[5(k – log 2 n )] E = Efficiency = S/2n = (2kn2)/[5(k – log 2 n ) (2n)] Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 96 Speedup and Efficiency With Pipelining

96
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 97 Speedup and Efficiency With Pipelining Speedup

97
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 98 Speedup and Efficiency With Pipelining Speedup Efficiency

98
Technique Can Work For Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 99 Second-order linear recurrences x 0 = a 0 x 1 = a 1 + b 1 x 0 x 2 = a 2 + b 2 x 1 + c 2 x 0... x n-1 = a n-1 + b n-1 x n-2 + c n-1 x n-3 Higher order linear recurrences

99
Technique Can Work For Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 100 Quotients of linear recurrences x 0 = a 0 x i = (a i + b i x i-1 )/(c i + d i x i-1 ) i=1,2,…, n-1 Other recurrences

100
Summary and Conclusions Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 101

101
Summary and Conclusions Chip technology is going to get even better/faster/cheaper for the foreseeable future. Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 102

102
Summary and Conclusions Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 103 Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)

103
Summary and Conclusions Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!) More massively parallel processing systems are going to be built and will become even better/faster/cheaper. Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 104

104
Summary and Conclusions Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 105 Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!) More massively parallel processing systems are going to be built and will become even better/faster/cheaper. The challenging world of parallel algorithmic design beckons and awaits creative minds.

105
Summary and Conclusions Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!) More massively parallel processing systems are going to be built and will become even better/faster/cheaper. The challenging world of parallel algorithmic design beckons and awaits creative minds. Yes, this means you! Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 106

106
Where in the World is Clemson University? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 107 We are here! Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011

107
Thank you for your kind attention! Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 108 Questions?

108
Extra Slides Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 109

109
Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 110 Links 1.Fujitsu K Computer (K = kei = Japanese word for 10 quadrillion) http://www.fujitsu.com/global/about/tech/k/ http://en.wikipedia.org/wiki/K_computer 2. NUDT Tianhe-1A Computer http://blog.zorinaq.com/?e=36 http://en.wikipedia.org/wiki/Tianhe-I 3. Cray Jaguar http://en.wikipedia.org/wiki/Jaguar_(computerhttp://en.wikipedia.org/wiki/Jaguar_(computer) http://www.nccs.gov/jaguar/ 4. Dawning Nebulae http://en.wikipedia.org/wiki/Dawning_Information_Industry http://www.theregister.co.uk/2010/05/31/top_500_supers_jun2010/ 5. NEC/HP Tsubame 2.0 http://en.wikipedia.org/wiki/TOP500

110
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 111 0 -Degree

111
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 112 0 -Degree

112
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 113 0 -Degree

113
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 114 0 -Degree

114
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 115 0 -Degree 1 -Degree 0 1

115
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 116 0 -Degree 1 -Degree 0 1 01

116
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 117 0 -Degree 1 -Degree 0 1 01 01

117
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 118 0 -Degree 1 -Degree 0 1 01 01

118
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 119 0 -Degree 1 -Degree 0 1 0001 1011 2 -Degree

119
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 120 3 -Degree 0 -Degree 1 -Degree 0 1 0001 1011 2 -Degree

120
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 121 3 -Degree 4 -Degree 0 -Degree 1 -Degree 0 1 0001 1011 2 -Degree

121
Hypercube Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 122 3 -Degree 4 -Degree 5 -Degree

122
All-to-All Communication (Hypercube) Roy Pargas, Clemson University pargas@clemson.edu July 30, 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines 123 ProblemMotivation

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google