1. Models and Languages for Parallel Computation
D. Skillicorn and D. Talia,
ACM Computing Surveys, Vol. 30, No. 2, June 1998
Presented by
Russell Zuck
June 28, 2005

2. Overview Motivation Difficulties of parallel programming
Evaluation criteria
Categories of parallel programming models
Summary of trends in model development
A related perspective and observation

3. Motivation What programming models are out there?
How well do they help solve some of the problems inherent with parallel programming?
How difficult are they to use?
What type of architecture is it used on?

4. Difficulties of Parallel Programming
Many architectures
Most are special purpose
No standards
Many languages/tools
Optimizers don’t work well
Wide range of programming paradigms
Lack of skilled programmers
Parallel thinking
Lack of a substantial market

5. Evaluation Criteria Easy to program Software Development Methodology
Decomposition
Mapping
Communication
Synchronization
Software Development Methodology
How do you determine correctness?
Sequential techniques of testing and debugging don’t extend well to parallel systems
Large state space to test due to extra degree of freedom
Testing limited to a couple of architectures

6. Evaluation Criteria Architecture independence Easy to understand
Can a large number of people become proficient at it?
Guaranteed performance
How much will execution performance differ from one architecture to another?
Cost Measures
Execution time
Processor utilization
Development costs

7. Parallel Programming Models
Differ in the degree of abstraction of concepts from the programmer
Parallelism
Decomposition
Mapping
Communication
Synchronization
The ideal model
The programmer is not required to be explicitly involved in any of the above operations

9. Nothing Explicit Complete abstraction from parallel operations
These exist only as “we are nearly there” types of languages
Examples
Optimizing compilers
Haskell: High order functional language

10. Parallelism Explicit Techniques for identifying parallelism required
Library functions
Syntax extensions
Example
Fortran

11. Decomposition Explicit
Programmer must specify the division of the problem into parallelizable pieces
Library functions
Example
BSP (Bulk synchronous parallelism)

12. Mapping Explicit
Programmer must specify the distribution of program pieces
Example
RPC
Paralf: An annotated functional language

13. Communication Explicit
Programmer responsible for interprocessor communication
Example
ABCL/1

14. Everything Explicit Almost no abstraction from the programmer Examples
Orca
PRAM
MPI

15. Summary of Trends So which is the best compromise?
As with most things, the middle of the road seems to be the best
A model with a medium amount of programmer abstraction
Trends in model development
Work on models with little abstraction is diminishing
Concentration of effort on models with midrange abstraction. Conceal some aspects of parallelism while preserving expressiveness
Some hope still resides with highly abstract models…Don’t Hold Your Breath Too Long!!!

16. Another Perspective Speculation of the future Alternate solution
As parallel machines regain popularity, more manufactures will be willing to produce them
Eventually a handful of standard architectures will emerge to cover SIMD, MIMD (Shared Memory), and MIMD (Fixed Connection)
Alternate solution
Use a one or two development languages and develop virtual machines (middle-ware) for each type of architecture
Similar to the Java paradigm

17. Programming Language
VM1
VM2
VM3
MIMD (Shared)
MIMD (Fixed)
SIMD