1. RUNNING PARALLEL APPLICATIONS BEYOND EP WORKLOADS IN DISTRIBUTED COMPUTING ENVIRONMENTS Zholudev Yury

2. Traditional desktop grids Scalable Distributed Dynamic Heterogeneous Centralized tasks pool Little or no communications between client-side tasks

3. Traditional desktop grids messaging between computational nodes could be implemented via server-side communications

4. Traditional desktop grids messaging between computational nodes could be implemented via server-side communications yields significant amount of synchronization overhead handled mostly on server side

5. Traditional desktop grids messaging between computational nodes could be implemented via server-side communications yields significant amount of synchronization overhead handled mostly on server side reduces reliability and performance

6. Peer-to-peer desktop grids Decentralized Scalable Dynamic Internode communications Network topology independence High latency Hard to control Discovery and routing overhead

7. Peer-to-peer desktop grids Jalapeno, JNGI, OurGrid, Triana, Xeerkat Distributed resource management and sequential tasks (bag-of- tasks) deployment using P2P-based file sharing Partially support EP applications P3 (Personal Power Plant) Master-Worker and Message Passing programming models for static set of nodes using JXTA P2P-MPI MPJ implementation for dynamic set of nodes located inside single LAN with parallel processes replication using number of super- nodes Cohesion Orbweb (SAT solving) Implements and employs distributed tasks pool for a subset of IS problems using XMPP

8. The goal Implement a light-weight p2p-based parallel computing middleware featuring: Network topology independence Reliability Support for Message Passing programming model

9. Requirements Network topology independence Parallel task can be submitted from any participating peer Communications between parallel task processes Support for simultaneous run of multiple tasks Result data can be accessed via any participating peer Support for dynamic sets of peers Parallel task may run on any number of peers Disconnection of any peer should not result in overall failure Processes can migrate between peers Processes can have multiple replicas running on different peers Multiple platforms support

10. Programming model Parallel task – set of interacting parallel processes (jobs) Each process can address any other process via messaging Parallel task execution is split into computation regions by task-wide control points Communication operations are carried out at control points Processes are topology- oblivious and address other processes via their numbers

11. Implementation: peer cycle

12. Implementation: architecture Java Cross-platform Ease of deployment JXTA (JXSE 2.6) Discovery services Routing Content transfer services Pipes and Sockets Custom objects advertisements Peer group abstractions and membership job process file system file service messaging service job and messages tracking and migration JXTA peer group file transfer service messaging service discovery services network

13. Current work: X-P2P Basic prototype implementation with fixed number of jobs per application Jobs migration and replication strategies Random job stealing/pushing Failure detection and recovery strategies Security Peer group membership Topology and network performance awareness Application deployment Jobs grouping Resource management Experiments and performance evaluation

14. Ultimate goals Meet leading supercomputers performance on Linpack Make it to the top of Graph500

15. THANK YOU Contact & participate! code.google.com/p/xp2p/ diemenator@gmail.com