1. Tammy Dahlgren, Tom Epperly, Scott Kohn, & Gary Kumfert

2. GKK 2 CASC Format Recap of Gatlinburg Talk...... only in Reverse...... Conceptually, not literally.

3. GKK 3 CASC Example #1: 1-D Vectors 12 0 345 678 910 11 0.0 1.1 2.2 3.3 4.4 5.5.9.8.7.6.5.4 0 0 1 2 3 4 5 1 2 3 4 5 double d = x.dot( y ); x x y y y

4. GKK 4 CASC Example #2: Undirected Graph 12 0 3 12 0 345 678 910 11 678 910 11 12 0 345 45 67 8 910 11

5. GKK 5 CASC Big Picture l What is the largest set of MxN Redistributable components? l How do we allow developers to add to this set? l Generality over Performance

6. GKK 6 CASC Other Complicated Tasks in CS l GUI APIs u Java-AWT: must extend “Frame” u Lots of default behaviors inherited l Threaded Apps u Java: class must implement Runnable u Must implement methods so JVM can interact with class correctly

7. GKK 7 CASC Last few Rhetorical Questions l Can we employ a similar strategy for MxN? l Corollaries u If so, which one? u How?

8. GKK 8 CASC Tale of two Customers l Library Developer l Familiar with Component Tech. l Formal Software Training l Application Programmer l May not be aware of using Babel l May have no formal SW training Developer User

9. GKK 9 CASC Zen of Babel l If its difficult...... let Babel do it. l If its impossible for Babel...... let developer do it. l If its difficult for user...... shift burden to Babel or Developer if at all possible

10. GKK 10 CASC Solution l Babel provides a MUX in its runtime l Parallel Distributed Components MUST IMPLEMENT the MxNRedistributable Interface l All communication & data redistribution details hidden from user. except performance, of course! ;-)

11. GKK 11 CASC Ramifications l User still programs in SPMD model l User may not selectively exclude processes or threads from a parallel RMI l User may not alter communication modes (synch v. asynch, etc.) u Determined by Babel u Modifiers may be available to developer in SIDL... e.g. “oneway” !

12. GKK 12 CASC Ramifications (2) l No MPI communicator connecting M to N u May not know M and N when driver is launched l M = N, M N, M=1, N=1, Okay l Non-rectilinear Data, No Problem l Owner Computes

13. GKK 13 CASC Now Drilling One Level Deeper...

14. GKK 14 CASC Babel Inserts Code between User & Developer l language interoperability u COMPLEX*(4) == complex l virtual function dispatch u call foo() on interface, dispatch to implementing class l implement RMI l remap distributed data no industry precedent

15. GKK 15 CASC Impls and Stubs and Skels Application Stubs Skels IORs Impls l Application: uses components in user’s language of choice l Client Side Stubs: translate from application language to C l Internal Object Representation: Always in C l Server Side Skeletons: translates IOR (in C) to component implementation language l Implementation: component developers choice of language. (Can be wrappers to legacy code)

16. GKK 16 CASC Out of Process Components Application Stubs IPC IORs Skels IORs Impls IPC

17. GKK 17 CASC Internet Remote Components Application Stubs Marshaler IORs Line Protocol Unmarshaler Skels IORs Impls

18. GKK 18 CASC All MxN Components inherit one interface l All Distributed Objects are Containers l They are by nature subdivisible l If an interface can make any container “look like” a 1-D vector then a MUX is little more than a KELP Mover

19. GKK 19 CASC MxNRedistributable Interface interface Serializable { store( in Stream s ); load( in Stream s ); }; interface MxNRedistributable extends Serializable { int getGlobalSize(); local int getLocalSize(); local array getLocal2Global(); split ( in array maskVector, out array pieces); merge( in array pieces); };

20. GKK 20 CASC We saw something similar yesterday l Kelp uses callbacks for user to define copy, serialization, etc. l I’m using an interface that user must implement l Kelp uses rectilinear regions l I’m using explicit local-global maps (for now) l Kelp has a Mover u most of the “guts” of the solution u that depends on the user callbacks l I’m calling it a MUX

21. GKK 21 CASC MxNRedistributable Interface in action l Last Time u I started with object creation u I handled problems as they aroze l This time u I’ll just do the MxN stuff u I’ll gloss over details altogether

22. GKK 22 CASC Example #2: Undirected Graph 12 0 3 12 0 345 678 910 11 678 910 11 12 0 345 45 67 8 910 11

23. GKK 23 CASC pp->minSpanTree( graph ); 12 0 345 6 12 0 345 678 910 11 2 345 678 910 5 678 9 11 orb graph MUX graph MUX pp l MUX queries graph for global size (12) l Graph determines particular data layout (blocked) 012345012345 6 7 8 9 10 11 require l MUX is invoked to guarantee that layout before render implementation is called

24. GKK 24 CASC MUX generates communication schedules ( client & server ) 12 0 345 6 12 0 345 678 910 11 2 345 678 910 5 678 9 11 orb graph MUX graph MUX pp 012345012345 6 7 8 9 10 11 require 0, 1, 2, 3 4, 5 6, 7 8, 9, 10, 11

25. GKK 25 CASC MUX opens communication pipes 12 0 345 6 12 0 345 678 910 11 2 345 678 910 5 678 9 11 orb graph MUX graph MUX pp 012345012345 6 7 8 9 10 11 require 0, 1, 2, 3 4, 5 6, 7 8, 9, 10, 11

26. GKK 26 CASC MUX splits graphs with multiple destinations (server-side) 12 0 345 6 12 0 345 678 910 11 2 345 678 910 5 678 9 11 orb graph MUX graph MUX pp 012345012345 6 7 8 9 10 11 require 0, 1, 2, 3 4, 5 6, 7 8, 9, 10, 11 2 345 678 345 678 910

27. GKK 27 CASC MUX sends pieces through communication pipes (persistance) 12 0 345 6 12 0 345 678 910 11 2 345 678 910 orb graph MUX graph MUX pp 012345012345 6 7 8 9 10 11 require 5 678 910 11 2 345 678 345 678 910 12 0 345 6 5 678 9 11

28. GKK 28 CASC MUX receives graphs through pipes & assembles them (client side) 12 0 345 6 12 0 345 678 910 11 2 345 678 910 orb graph MUX graph MUX pp 012345012345 6 7 8 9 10 11 require 5 678 910 11 2 345 678 345 678 910 12 0 345 6 5 678 9 11 345 678 910 11 12 0 345 678

29. GKK 29 CASC Summary l All distributed components are containers and subdivisable l The smallest globally addressable unit is an atom l MxNRedistributable interface reduces general component MxN problem to a 1-D array of ints l MxN problem is a special case of the general problem N handles to M instances l Babel is uniquely positioned to contribute a solution to this problem

30. GKK 30 CASC Tentative Research Strategy l Java only, no Babel u serialization & RMI built-in l Build MUX l Experiment l Write Paper l Finish 0.5.x line l add serialization l add RMI l Add in technology from Fast Track Fast TrackSure Track

31. GKK 31 CASC Closing Remarks l User calls without regard to data distribution l Developers code assuming data distributed appropriately l Babel does all the redistribution between the two u Requires Developers implementing an interface (a.k.a. callbacks)

32. GKK 32 CASC Open Questions l Non-general, Optimized Solutions l Client-side Caching issues l Fault Tolerance l Subcomponent Migration l Inter vs. Intra component communication l MxN, MxP, or MxPxQxN

33. GKK 33 CASC MxPxQxN Problem Long-Haul Network

34. GKK 34 CASC MxPxQxN Problem Long-Haul Network

35. GKK 35 CASC The End

36. GKK 36 CASC Work performed under the auspices of the U. S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under Contract W-7405-Eng-48 UCRL-VG-143383 26 Apr 2001