1. Real-World Distributed Computing with Ibis Henri Bal bal@cs.vu.nl Vrije Universiteit Amsterdam

2. Outline ● Distributed systems ● Distributed applications ● The Ibis distributed programming system ● Ibis in the mobile networked world

3. Distributed Systems: 1980s ● Networks of Workstations (NOWs) ● Collections of Workstations (COWs) ● Processor pools (Amoeba) ● Condor pools ● (Beowulf) clusters

4. Distributed Systems: 1990s ● Metacomputing (Smarr & Catlett, CACM) ● Flocking Condor (Epema) ● DAS (Distributed ASCI Supercomputer) ● Grid Blueprint (Foster & Kesselman) ● Desktop grids, SETI@home DAS-3

5. Distributed Systems: 2000s ● Cloud computing ● Infrastructure as a service ● Virtualization ● Mobile computing ● Sensor networks ● Smart phones ● The Networked World

6. Problem ● How to write high-performance applications for real-world distributed systems? ● How to integrate many different resources?

7. Our approach ● Study fundamental underlying problems ● … hand-in-hand with realistic applications ● … integrate solutions in one system: Ibis Distributed SystemsUser !

8. Fundamental problems ● Performance – efficiency on wide-area systems ● Heterogeneity – different systems & APIs ● Malleability – resources come and go ● Fault tolerance - crashes ● Connectivity – firewalls, NAT, etc.

9. Applications ● Scientific applications ● Imaging (VU Medical Center, AMOLF) ● Bioinformatics (sequence analysis, cell modeling) ● Astronomy (data analysis challenge) ● Multimedia content analysis ● Games and model checking ● Semantic web (distributed reasoning)

10. Awards SCALE 2008 (CCGrid’08) DACH 2008 – BS DACH 2008 - FT AAAI-VC 2007 ISWC 2008 Multimedia Computing Astronomy Semantic Web (van Harmelen et al.) (Cluster/Grid’08)

11. Multimedia content analysis ● Automatically extract information from images & video ● Extract feature vectors from images ● Describe properties (color, shape) ● Data-parallel task on a cluster ● Compute on consecutive images ● Task-parallelism on a grid

12. MMCA ‘ Most Visionary Research’ award at AAAI 2007, (Frank Seinstra et al.)

13. Games and Model Checking ● Can solve entire Awari game on wide-area DAS-3 ● Needs 10G private optical network (StarPlane) ● Distributed model checking has very similar communication pattern ● Search huge state spaces, random work distribution, bulk asynchronous transfers ● Can efficiently run DeVinE model checker on wide-area DAS-3, use up to 1 TB memory ● See IPDPS’09 (May 2009)

14. Distributed reasoning ● MaRVIN (Frank van Harmelen et al, VU): ● a distributed platform for massive RDF inferencing (deductive closure) ● ``a brain the size of a planet’’ ● Uses Ibis to run on heterogeneous systems (clusters, desktop grids) ● 3rd prize at Billion Triple track of Semantic Web Challenge 2008

15. European users ● D-Grid: Workflow engine for astronomy ● U. Erlangen: grid file system ● INRIA: ProActive on Ibis RMI ● U. Patras: Jylab scientific computing ● UPC Barcelona: Grid Superscalar ● HITACHI: Peta-scale data management Grid’5000

16. Outline ● Distributed systems ● Distributed applications ● The Ibis distributed programming system ● Ibis in the mobile networked world

17. Ibis Philosophy ● Real-world distributed applications should be developed and compiled on a local workstation, and simply be launched from there

18. Ibis Approach ● Virtual Machines (Java) deal with heterogeneity ● Provide range of programming abstractions ● Designed for dynamic/faulty environments ● Easy deployment through middleware- independent programming interfaces ● Modular and flexible: can replace Ibis components by external ones

19. Ibis Design ● Functionality from programming languages ● High-Performance Application Programming System ● Functionality from operating systems ● Distributed Application Deployment System

20. Ibis System

21. Programming system ● Programming models: ● Message passing (RMI, MPJ) ● Divide-and-conquer (Satin) ● Jorus: (multimedia applications) ● Dataflow framework (Maestro)  see HPDC09 Sat 14.30 ● IPL (Ibis Portability Layer) ● Java-centric “run-anywhere” library ● Point-to-point, multicast, streaming, …. ● Simple model (Join-Elect-Leave ) for tracking resources, supports malleability & fault-tolerance

22. SmartSockets library ● Detects connectivity problems ● Tries to solve them automatically ● With as little help from the user as possible ● Integrates existing and several new solutions ● Reverse connection setup, STUN, TCP splicing, SSH tunneling, smart addressing, etc. ● Uses network of hubs as a side channel

23. Example

25. Deployment system ● IbisDeploy GUI ● JavaGAT: ● Java Grid Application Toolkit ● Make applications independent of underlying middleware ● Zorilla P2P system ● Job management, gossiping, clustering, flood scheduling

26. JavaGAT Grid Application File.copy(...)‏ submitJob(...)‏ cp ftp gridftp scp http fork pbs condor unicore globus ● For simple grid operations there are many ways of implementing them ● different middleware available on different sites

27. JavaGAT Grid Application File.copy(...)‏ submitJob(...)‏ cp ftp gridftp scp http fork pbs condor unicore globus ? ? ● Which should you use ? ● Some may not be available on all sites ● Many combinations

28. JavaGAT Grid Application File.copy(...)‏ submitJob(...)‏ GAT Engine Remote Files Monitoring Info service Resource Management GridLabGlobusUnicoreSSHP2PLocal GAT gridftpglobus

29. Multimedia Content Analysis Client Broker Servers Ibis (Java) ● Runs simultaneously on clusters (DAS-3, Japan, Australia), Desktop Grid, Amazon EC2 Cloud ● Connectivity problems solved automatically by Ibis SmartSockets

30. Connection management With SmartSockets: run everywhere Standard sockets: only local VU machines can be reached due to firewalls problems

31. Ibis movie (part 1)

32. Performance on 1 DAS-3 cluster ● Relative speedups of Java/Ibis and C++/MPI ● Using TCP or Myricom’s MX protocol ● Sequential performance Java: 80% of C++

33. Speedup (wide-area) ● Homogeneous wide-area systems (DAS-3): ● Frame rate increases linearly with #clusters ● World-wide experiment : ● 24 frames per second ● Speed limited by camera, not computing infrastructure

34. Outline ● Distributed systems ● Distributed applications ● The Ibis distributed programming system ● Ibis in the mobile networked world

35. Smart Phones ● GSM + PC + GPS + camera + networks + …. ● Location-aware ● What if everyone always carries a smart phone (like a GSM now)? ● Next wave in computing?

36. Ibis on Smart Phones ● Our focus: distributed smart phone applications ● Applications running on multiple phones ● Integration with distributed computing backbone ● Use Android for development ● Google’s open-source platform ● Java-based

37. Distributed applications ● Disaster management (Katrina) ● Use ad-hoc Wifi network when GSM network fails ● Finding nearby people with certain skills ● Bus drivers, CPR ● Distributed decision support ● Moving people to shelters (logistics) ● Social networks ● Similar issues ● Find nearby friends, decide on restaurant

38. Wild example ● Track position => automatic diary of your life ● Cross-comparisons between diaries Haven’t we met before? Yes, on 23 Oct 2010, 3.48 pm at N 52°22.688´ E 004°53.990´

39. eyeDentify ● Object recognition on a G1 smartphone ● Smartphone is a limited device: ● Can run only 64 x 48 pixels (memory bound) ● 1024 x 768 pixels would take 5 minutes ● Distributed Ibis version: + = + 1024 x 768 pixels 2.0 seconds

40. Ibis movie (part 2)

41. Interdroid Distributed Communication Data Management Novel Mobile Distributed Applications Context Sensitive Programming Models

42. Current work ● Raven: API for Viable Episodic Networking ● Decentralized synchronization API ● Fine grained control over data sharing ● Bluetooth support for ad-hoc communication ● Discovery of devices using multiple networks ● Context Aware Programming Models ● Supporting distributed decision making ● Representing and using context (location etc.) ● Exploiting social relationships (Hyves, Facebook)

43. Summary ● Ibis provides integrated solutions for many hard problems ● performance, heterogeneity, malleability, fault tolerance, connectivity ● It combines functionality from programming languages and operating systems ● Used for many applications on real-world distributed systems ● Download from http://www.cs.vu.nl/ibis/

44. Acknowledgements Niels Drost Ceriel Jacobs Roelof Kemp Timo van Kessel Thilo Kielmann Jason Maassen Rob van Nieuwpoort Nick Palmer Kees van Reeuwijk Frank J. Seinstra Kees Verstoep Gosia Wrzesinska

45. Questions?

46. DAS-3DAS-3