1. Parallel Computing Demand for High Performance
ITCS 4/5145 Parallel Programming UNC-Charlotte, B. Wilkinson January 3, 2016

2. Parallel Computing Motives
Using more than one computer, or a computer with more than one processor, collectively to solve a problem.
Motives
Usually faster computation.
Very simple idea
n computers operating simultaneously can achieve the result faster
it will not be n times faster for various reasons
Other motives include: fault tolerance, larger amount of memory available, ...

3. Parallel programming has been around for more than 50 years
Parallel programming has been around for more than 50 years. Gill writes in 1958*:
“... There is therefore nothing new in the idea of parallel programming, but its application to computers. The author cannot believe that there will be any insuperable difficulty in extending it to computers. It is not to be expected that the necessary programming techniques will be worked out overnight. Much experimenting remains to be done. After all, the techniques that are commonly used in programming today were only won at the cost of considerable toil several years ago. In fact the advent of parallel programming may do something to revive the pioneering spirit in programming which seems at the present to be degenerating into a rather dull and routine occupation ...”
* Gill, S. (1958), “Parallel Programming,” The Computer Journal, vol. 1, April, pp

4. Problems needing multiple computers together to be solved
The key aspect is the execution time.
Sometimes there is a strict deadline for the solution, for example real time applications, flight control, …
Sometimes is deadline is less quantifiable, but still very important, for example:
CT scan image processing. A doctor needs that quickly within minutes or less to work effectively.
Computer assisted engineering design becomes ineffective if each step takes a long time.
Sometimes present day computers simply cannot produce the result at all in a reasonable time -- so-called “Grand Challenge” problems, e.g. global weather forecasting, modeling large number of interacting bodies, large DNA structures …

5. Weather Forecasting
Atmosphere modeled by dividing it into 3-dimensional cells.
Calculations of each cell repeated many times to model passage of time.
Temperature, pressure, humidity, etc.

6. Modeling Motion of Astronomical Bodies
Each body attracted to each other body by gravitational forces.
Movement of each body predicted by calculating total force on each body and applying Newton’s laws (in the simple case) to determine the movement of the bodies.

7. Modeling Motion of Astronomical Bodies
Each body has N-1 forces on it from the N-1 other bodies - O(N) calculation to determine the force on one body (three dimensional).
With N bodies, approx. N2 calculations, i.e. O(N2) *
After determining new positions of bodies, calculations repeated, i.e. N2 x T calculations where T is the number of time steps, i.e. the total complexity is O(N2T).
* There is an O(N log2 N) algorithm, which we will cover in the course

8. A galaxy might have, say, 1011 stars.
Even if each calculation done in 1 ms (extremely optimistic figure), it takes:
109 years for one iteration using N2 algorithm or
Almost a year for one iteration using the N log2 N algorithm assuming the calculations take the same time (which may not be true).
Then multiple the time by the number of time periods!
We may set the N-body problem as an assignment using the basic O(N2) algorithm. However, you do not have 109 years to get the solution and turn it in.

9. My code with graphics running on coit-grid server and forwarded to client PC.
See video for demo

10. Questions