Patterns for Parallel Programming Markos Chandras Salil Ponde Wenjiang Xu COMP60611: Fundamentals of Parallel and Distributed Systems (2010) School of Computer Science University of Manchester

Distributed data moving mechanisms Reduced synchronization overhead Fine-grain parallelism Difficulties in construction and managment of memories for data matching Load balancing problems in large systems Input/output data define the task interface Hidden implementation details No relationship among tasks is explicitly defined The scheduler manages the tasks via the interface Optimized scheduling using data packing,, data reuse Fault tolerated, task migration References The 'data-driven' model 1) Dan I. Moldovan (1993) - ”'Parallel Processing – From Applications to Systems” (page 399) 2) V. D. Tran, L. Hluchý, G. T. Nguyen - “Parallel Programming with Data Driven Model” ( Parallel and Distributed Processing, Proceedings. 8th Euromicro Workshop, Rhodos, Greece, 2000 )

Dealing effectively with large arrays Array elements should be close to the Processing Element (PE) which processes it Ways of distribution (a) One Dimensional (b) Two Dimesional (c) Block Cyclic Problem is defined in terms of global array indices Program is written in terms of local array indices Choosing the way of distribution References The Distributed Array Pattern 1) Timothy G. Mattson; Beverly A. Sanders; Berna L. Massingill (2004) - ”'Patterns for Parallel Programming”

Semi-independent tasks interact in an irregular fashion. (a) interaction is determined by the flow of data (b) the data implies the ordering constrains between the task (c) direct graph with cycles Solution: events & tasks (a) Task: receive event-->process event -->(possibly) send events (b) Event Flow: need a form of asychronous communication of events message-passing: from task that generate the event to task that process it (c) Avoiding deadlock (1) more informational (2) timeouts (d) One task per PE or Multiple task (need good load balance) (e) Communication if events should be efficient References The Event-Based Coordination Pattern 1) Timothy G. Mattson; Beverly A. Sanders; Berna L. Massingill (2004) - ”'Patterns for Parallel Programming”

List of References Dan I. Moldovan (1993). Parallel Processing – From Applications to Systems. Morgan Kaufmann Publishers Inc. San Francisco, CA, USA V. D. Tran, L. Hluchý, G. T. Nguyen Parallel Programming with Data Driven Model. Parallel and Distributed Processing, Proceedings. 8th Euromicro Workshop, Rhodos, Greece, 2000 Timothy G. Mattson, Beverly A. Sanders, Berna L. Massingill (2004). Patterns for Parallel Programming. Addison-Wesley