1. An Integrated Synchronization and Consistency Protocol for the Implementation of a High-Level Parallel Programming Language
Martin C. Rinard
University of California, Santa Barbara
Santa Barbara, California 93106

2. Context Parallel Programming System
Single Address Space Programming Model
Message Passing Machine
Key Functionality
Synchronization
Communication
Consistency
Straightforward Implementation Separates Protocols
Synchronization Messages
Communication and Consistency Messages
Redundant Messages

3. High-Level, Implicitly Parallel Language
Jade
High-Level, Implicitly Parallel Language
Single Address Space
Serial Semantics
Portable
Shared Memory Machines - Stanford DASH
Message Passing Machines - iPSC/860, CM-5
Heterogeneous Networks of Workstations
Message Passing Implementation
Combines Synchronization and Consistency Protocols
Result: Single Efficient Protocol

4. Structure of Talk Jade Programming Model Synchronization Mechanism
Integrated Synchronization and Consistency Protocol
Experimental Results

5. Programming Model Programmer Starts With A Serial Program
Identifies Data Granularity using Shared Objects
Identifies Computation Granularity using Tasks
Provides Access Specification For Each Task
Jade Implementation Analyzes Access Specifications
Automatically Extract Parallelism
Synchronize Computation
Generate Communication
Ensure Consistency of Shared Objects
Preserves Serial Semantics

6. Starts With Serial Program
Programming Model
Starts With Serial Program
Data Granularity - Shared Objects
Task Granularity
Data Usage Information
Extracts Concurrency
Synchronizes Computation
Generates Communication
Ensures Consistency of Shared Objects
Programmer
Implementation

7. Concurrency Principle
Tasks Execute Serially if
One declares a write to a piece of data
Other declares either read or write to the same piece of data
Same execution order as serial program
Otherwise, Tasks Execute Concurrently rd wr rd wr wr wr rd rd wr wr wr

8. Key Aspects Implementation Manages Concurrency Synchronization
Communication
Consistency
Promotes Modularity
Serial Semantics
Deterministic Execution
Only Serial Programs
Portable

9. Object Queues wr rd wr rd wr wr rd rd wr wr

10. Object Queues wr rd rd wr rd rd rd rd Time wr rd wr rd wr wr rd wr rd

11. Consistency Problem Processor 0 Processor 1 Time Invalidate Protocolwr
Time wr rd wr rd
Invalidate Protocol
Update Protocol
Expensive Global Operations

12. Consistency Mechanism
Object Replica Has Version Number
Object Queue Maintains
Latest Version Number
Owner of Latest Version
Queue Tells Each Task
What Version to Access
Owner of that Version
Enabled Task Checks Version Numbers
If Obsolete, Fetch From Owner
If Up to Date, No Fetch

13. Consistency Mechanism
Processor
Processor 1 wr rd wr rd wr 2 2 2 wr rd rd wr rd rd wr wr 3 2 3 rd rd wr rd wr rd 3 2 3 rd rd wr rd wr rd 3 3 3 rd rd

14. Result A More Efficient Consistency Mechanism
No Required Invalidate Messages
No Required Update Messages
Enabling Features
High-Level Language Model
Data Usage Information
Integration With Synchronization Mechanism

15. Experimental Results Compare Three Consistency Mechanisms
Update (Broadcast if all Processors should receive Object)
Invalidate
Version
Applications
Panel Cholesky
Ocean
Water
Collected Data on iPSC/860
Communication Volume
Number of Messages

16. Panel Cholesky
Number of Messages
Communication Volume

17. Ocean
Number of Messages
Communication Volume

18. Water
Number of Messages
Communication Volume

19. Adaptive Broadcast Goal
Advantages of Broadcast for Widely Accessed Data
Without Disadvantages of Update Protocol
Policy
If Every Processor Reads Same Version of Object
Jade Implementation Uses Broadcast Protocol for that Object
Result
Performs as well as Update Protocol for Water
Performs as well as Version for Panel Cholesky and Ocean

20. Related Work Consistency Based on Version Numbers NAMOS (Reed)
Sprite (Nelson, Welch, Ousterhout)
SAM (Scales, Lam)
VDOM (Feeley, Levy)
TreadMarks (Keleher, Dwarkadas, Cox, Zwaenepoel)
Midway (Zekauskas, Sawdon, Bershad)

21. Conclusion Jade Presents High-Level Abstractions Serial Semantics
Single Address Space
Jade Implementation Provides an Integrated Protocol
Synchronization
Communication
Consistency
Experimental Results
Adaptive Broadcast