1. June 1998 CHAIMS1 Compiling High-level Access Interfaces for Multi-site Software Stanford University Objective: Investigate revolutionary approaches to large-scale software composition. Approach: Develop and validate a composition-only language. Planned contributions: Asynchrony by splitting up CALL-statement. Hardware and software platform independence. Potential for multi-site dataflow optimization. Performance optimization by invocation scheduling. CHAIMS: Mega-Programming Research

2. June 1998 CHAIMS2 Participants Support –DARPA ISO EDCS program (1996-1999) –Siemens Corporate Research (1996-1998) –DoD AFOSR AASERT student support (1997-1999) –Sloan Foundation - computer industry study (1996-97) People - Gio Wiederhold PI -- Marianne Siroker: Administration –Dorothea Beringer (postdoc EPF Lausanne) since Dec.1997 –Ron Burback (CS PhD cand.) –Laurence Melloul (CS MS) –Woody Pollack (CS MS) –MS and BS CS graduated: Joshua Hui, Gaurav Bhatia, Prasanna Ramaswami, Kirti Kwatra, Pankaj Jain, Mehul Bastawala, Catherine Tornabene, Wayne Lim, Connan King –Louis Perrochon (postdoc ETH Zurich) Fall quarter 1996

3. June 1998 CHAIMS3 My Personal Background Masters in Computer Science: hybrid-monitoring tool for debugging and software performance analysis for distributed software Software engineer: telecommunication systems Consultant: software methodologies, quality assurance, project management, CASE-tools PhD: Modeling scenarios in object-oriented analysis Teaching: Fusion Now: CHAIMS -- large-scale software composition, distributed systems

4. June 1998 CHAIMS4 Presentation Motivation and Objectives –changes in software production –basis for new visions and education Concepts of CHAIMS –CHAIMS language –CHAIMS architecture and composition process –Scheduling –Dataflow optimization Status, Plans, Conclusions

5. June 1998 CHAIMS5 Coding Integration 1970 1990 2010 Shift in Programming Tasks

6. June 1998 CHAIMS6 Typical Scenario: Logistics A general has to ship troops and/or various material from L.A. to Washington DC: –different kind of material: criteria for preferred transport differ –not every airport equally suited –congestion, prices –weather constraints –exact due or ready dates Today: calling different companies, looking up information on the web, reservations by hand Future: system proposes possibilities that take into account various conditions »hand-coded systems »composition of processes

7. June 1998 CHAIMS7 Scaling alternatives ?

8. June 1998 CHAIMS8 Megamodules Megaprogram for composition, written by domain programmer CHAIMS system automates generation of client for distributed system Megamodules, provided by various megamodule providers CHAIMS

9. June 1998 CHAIMS9 Megamodules - Definition Megamodules are large, autonomous, distributed, heterogeneous services or processes. large: computation intensive, data intensive, ongoing processes (monitoring services) distributed: to be used by more than one client heterogeneous: accessible by various distribution protocols (not only different languages and systems) autonomous: maintenance and control over recourses remains with provider, differing ontologies ( ==> SKC) Examples: –logistics: “find best transportation route from A to B”, reservation systems –genomics: easier framework for composing various processing tools than ad-hoc coding

10. June 1998 CHAIMS10 Challenge: Thin Clients (1) Domain expert Client computer Control & Computation Services I/O a b c d e Wrappers to resolve differences I/O Data Resources

11. June 1998 CHAIMS11 Challenge: Thin Clients (2) Domain expert Client workstation Computation Services IO module MEGA modules IO module a b c d e Data Resources Sites R T SU T C

12. June 1998 CHAIMS12 Challenge: Heavy-weight Services Services are not free for a client: execution time of a service transfer time for data fees for services What we need: ==> monitoring progress of a service ==> possibility to choose among equivalent services based on estimated waiting time and fees ==> parallelism among services ==> preliminary overview results, choosing level of accuracy / number of results for complex processes ==> novel optimization techniques

13. June 1998 CHAIMS13 Challenge: Non-technical Domain Experts Company providing services: domain experts of domain of service (e.g. weather) technical experts for programming for distribution protocols, setting up servers in a middleware system marketing experts “Megaprogrammer”: is domain expert of domain that uses these services is not technical expert of middleware system or experienced programmer, wants to focus on problem at hand (=results of using megaprogram) e.g. scientist, assistant of a general

14. June 1998 CHAIMS14 Challenge: Purely Compositional Language Possible? Which languages did succeed? –Algol, ADA: integrated composition and computation –C, C++: focus on computation Why new language? –complexity: not all facilities of a common language (compare to approach of Java), –inhibiting traditional computational programming (compare C++ and Smalltalk concerning object-oriented programming) –focus on issue of composition, parallelism by asynchrony, and optimization

15. June 1998 CHAIMS15 CHAIMS “Logical” Architecture Customer Megaprogram clients (in CHAIMS) Network/Transport (DCE, CORBA,...) Megamodules (Wrapped or Native)

16. June 1998 CHAIMS16 CHAIMS Physical Architecture Network DCE, CORBA, JAVA RMI, DCOM... Megaprogram Clients in CHAIMS Megamodules (wrapped, native) each supporting setup, estimate, invoke, examine, extract, and terminate.

17. June 1998 CHAIMS17 Decomposing CALL statements Copying Code sharing Parameterized computation Objects with overloaded method names Remote procedure calls to distributed modules Constrained (black box) access to encapsulated data progress in scale of computing Extract InvokeEstimateExamineSetup CHAIMS decomposes CALL functions CALL gained functionality

18. June 1998 CHAIMS18 CHAIMS Primitives Pre-invocation: SETUP: set up the connection to a megamodule SET-, GETATTRIBUTES: set global parameters in a megamodule ESTIMATE: get estimate of execution time for optimization Invocation and result gathering: INVOKE: start a specific method EXAMINE: test status of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation or a connection to a megamodule Control:Utility: WHILE, IFGETPARAM: get default parameters

19. June 1998 CHAIMS19 Megaprogram Example: Overview InputOutput - Input - Output RouteInfo - AllRoutes - CityPairList -... AirGround - CostForGround - CostForAir -... Routing - BestRoute -... RouteOptimizer - Optimum -... General I/O-megamodule »Input function takes as parameter a default data structure containing names, types and default values for expected input Travel information: »Computing all possible routes between two cities »Computing the air and ground cost for each leg given a list of city-pairs and data about the goods to be transported Two megamodules that offer equivalent functions for calculating optimal routes »Optimum and BestRoute both calculate the optimum route given routes and costs »Global variables: Optimization can be done for cost or for time

20. June 1998 CHAIMS20 Megaprogram Example: Code io_mmh = SETUP ("InputOutput") route_mmh = SETUP ("RouteInfo")... best2_mmh.SETATTRIBUTES (criterion = "cost") cities_default = route_mmh.GETPARAM(Pair_of_Cities) input_cities_ih = io_mmh.INVOKE ("input”, cities_default) WHILE (input_cities_ih.EXAMINE() != DONE) {} cities = input_cities_ih.EXTRACT()... route_ih = route_mmh.INVOKE ("AllRoutes", Pair_of_Cities = cities) WHILE (route_ih.EXAMINE() != DONE) {} routes = route_ih.EXTRACT() … IF (best1_mmh.ESTIMATE("Best_Route") < best2_mmh.ESTIMATE("Optimum") ) THEN {best_ih = best1_mmh.INVOKE ("Best_Route", Goods = info_goods, Pair_of_Cities = cities, List_of_Routes = routes, Cost_Ground = cost_list_ground, Cost_Air = cost_list_air)} ELSE {best_ih = best2_mmh.INVOKE ("Optimum", Goods = info_goods, …... best2_mmh.TERMINATE() // Setup connections to megamodules. // Set global variables valid for all invocations // of this client. // Get information from the megaprogram user // about the goods to be transported and about // the two desired cities. // Get all routes between the two cities. //Get all city pairs in these routes. //Calculate the costs of all the routes. // Figure out the optimal megamodule for // picking the best route. //Pick the best route and display the result. // Terminate all invocations

21. June 1998 CHAIMS21 Operation of one Megamodule SETUP SETATTRIBUTES provides context ESTIMATE serves scheduling INVOKE initiates remote computation EXAMINE checks for completion EXTRACT obtains results TERMINATE I / ALL M handle I handle M handle I handle

22. June 1998 CHAIMS22 CHAIMS Megaprogr. Language Purely compositional: –no primitives for arithmetic ==> math megamodules –no primitives for input/output ==> general and problem- specific I/O megamodules Splitting up CALL-statement: –parallelism by asynchrony in sequential program –novel possibilities for optimizations –reduction of complexity of invoke statements higher-level language (assembler => HLLs, HLLs => composition/megamodule paradigm)

23. June 1998 CHAIMS23 Architecture: Runtime e d a b c Distribution System (CORBA, RMI…) CSRT (compiled megaprogram) MEGA modules

24. June 1998 CHAIMS24 Architecture: Composition Process e d a b c MEGA modules CHAIMS Repository adds information to Megamodule Provider wraps non-CHAIMS compliant megamodules Wrapper Templates

25. June 1998 CHAIMS25 writes Architecture: Composition Process Megaprogrammer CSRT (compiled megaprogram) Megaprogram (in CHAIMS language) CHAIMS Compiler generates CHAIMS Repository information

26. June 1998 CHAIMS26 writes Architecture: Overview e Megaprogrammer d a b c Distribution System (CORBA, RMI…) CSRT (compiled megaprogram) Megaprogram (in CHAIMS language) CHAIMS Compiler generates MEGA modules CHAIMS Repository adds information to Megamodule Provider wraps non-CHAIMS compliant megamodules information Wrapper Templates

27. June 1998 CHAIMS27 writes Architecture: Overview e Megaprogrammer d a b c Distribution System (CORBA, RMI…) CSRT (compiled megaprogram) Megaprogram (in CHAIMS language) CHAIMS Compiler generates MEGA modules CHAIMS Repository adds information to Megamodule Providers wraps non-CHAIMS compliant megamodules information Wrapper Templates run-time

28. June 1998 CHAIMS28 Architecture: CHAIMS-Language and CHAIMS-Protocols Megaprogram Megaprogrammer M e g a m o d u l e s CHAIMS-language CHAIMS-protocols CORBA-idlDCE-idlJava-class CHAIMS API defines interface between megaprogrammer and megaprogram; the megaprogram is written in the CHAIMS language. The CHAIMS protocols define the calls the mega- modules have to understand. These protocols are slightly different for the different distribution protocols, and are defined by an idl for CORBA, another idl for DCE, and a Java class for RMI.

29. June 1998 CHAIMS29 Name of Person Architecture: Gentype A Gentype is a triple of name, type and value, where value is either a simple type or a list of other gentypes (i.e. a complex type). Possible simple types: given by ASN.1, the ASN.1- conversion library for C++, our own conversion routines. Example: Person_Information complex First Name string Joe Last Name string Smith Personal Data complex Address Date of Birthdate 6/21/54 Soc.Sec.No string 345-34-345

30. June 1998 CHAIMS30 Wrapper: CHAIMS Compliance CHAIMS protocol - knowing all CHAIMS primitives State management and asynchrony: »clientId (megamodule handle in CHAIMS language) »callId (invocation handle in CHAIMS language) »results must be stored for possible extraction(s) until termination of the invocation Data transformation: »all parameters of type blob (BER-encoded Gentype) must be converted into the megamodule specific data types (combination hand-coding/decoding routines

31. June 1998 CHAIMS31 Architecture: Three Views Transportation View moving around data blobs and CHAIMS messages Composition View (megaprogram) - composition of megamodules - directing of opaque data blobs Data View - exchange of data - interpretation of data - in/between megamodules CHAIMS Layer Distribution Layer Objective: Clear separation between composition of services, computation of data, and transport

32. June 1998 CHAIMS32 execution of a remote method synchronous invoke a method i e extract results setup / set attributes s s e i time decomposed (no benefit for one module) asynchronous s,i time e available for other methods e s,i Scheduler: Decomposed Execution

33. June 1998 CHAIMS33 Scheduler: Optimized Execution M1 M4 (<M1+M2) M5 M2 M3 (>M1+M2) i1 e1 e4 e3 e2 i3 i4 i5 i2 e5 time M1 M4 M5 M2 M3 i1 e1 e2 e3 e4 e5 i2 i3 i4 i5 time data dependencies execution of a method non-optimized, or handprogrammed optimized by scheduler according to estimates invoke a method i e extract results set up / set attributes (not shown) s

34. June 1998 CHAIMS34 Scheduling: Simple Example 1 cost_ground_ih = cost_mmh.INVOKE ("Cost_for_Ground", 1 List_of_City_Pairs = city_pairs,Goods = info_goods) 2 WHILE (cost_ground_ih.EXAMINE() != DONE) {} 3 cost_list_ground = cost_ground_ih.EXTRACT() 3 cost_air_ih = cost_mmh.INVOKE ("Cost_for_Air", 2 List_of_City_Pairs = city_pairs,Goods = info_good) 4 WHILE (cost_air_ih.EXAMINE() != DONE) {} 4 cost_list_air = cost_air_ih.EXTRACT() order in unscheduled megaprogram order in automatically scheduled megaprogram

35. June 1998 CHAIMS35 Scheduling: Possible Actions INVOKES: call INVOKE’s as soon as possible »may depend on other data »moving it outside of an if-block: depending on cost- function (ESTIMATE of this and following functions concerning execution time, dataflow and fees (resources). EXTRACT: move EXTRACT’s to where the result is actually needed »no sense of checking/waiting for results before they are needed »instead of waiting, polling all invocations and issue next possible invocation as soon as data could be extracted TERMINATE: terminate invocations that are no longer needed (save resources) »not every method invocation has an extract (e.g. print-like functions)

36. June 1998 CHAIMS36 Compiling into a Network Mega Program Module A Module B Module C Module E Module D Module F current CHAIMS system Mega Program Module D Module F control flowdata flow with automatic dataflow optimization Mega Program Module A Module B Module C Module E Module D Module F

37. June 1998 CHAIMS37 CHAIMS Implementation Specify minimal language –minimal functions: CALLs, While, If * –minimal typing {boolean, integer, string, handles, object} »objects encapsulated using ASN.1 standard –type conversion in wrappers, service modules* Compiler for multiple protocols (one-at-time, mixed*) Wrapper generation for multiple protocols Native modules for I/O, simple mathematics*, other Implement API for CORBA, Java RMI, DCE usage Wrap / construct several programs for simple demos Schedule optimization * Demonstrate use in heterogeneous setting * Define full-scale demonstration * in process

38. June 1998 CHAIMS38 Concept Status Definition of architecture for Megaprogramming –bottom up assessment of code to be generated examples: room reservation, shipping –primitives –handles for parallel operation –heterogeneity -- common features of distribution protocols Minimal language that can generate the code –no versus very few types -- ASN.1 for complex types –natural parallelism -- still a major research issue Awareness of novel optimizations –information flow constraints -- scheduling –direct data flow between megamodules

39. June 1998 CHAIMS39 Focus for Future Finishing basic infrastructure and demo examples. CHAIMS interpreter instead of CHAIMS compiler. Scheduling of invocations and extractions. Flexible interaction with megamodules; extracting and handling overview results. Direct dataflows between megamodules (future project).

40. June 1998 CHAIMS40 Conclusion: Research Questions Is a Megaprogramming language focusing only on composition feasible? Can it exploit on-going progress in client-server models and be protocol independent? Can natural parallelism for distributed services be effectively scheduled? Can high-level dataflow among distributed modules be optimized? Can CHAIMS express clearly a high-level distributed SW architecture? Can the approach affect SW process concepts and practice?

41. June 1998 CHAIMS41 Paying for SW Services You can not run an effective (SW) business and not be reimbursed for it. How? Four approaches: –Sell Softwaresell oilfield to customer –Lease copy / usage rightslease well –Time / user limited accessfill tank –Charge by use instanceprovide bus General problems, effects differ –IP protection? –keeping SW updated –billing for est.value –performance effect poorsomefairgood poorokgood simple awkw.hard no littlesome BuyLeaseLimitUse protect update bill perform

42. June 1998 CHAIMS42 Conclusion: Questions not addressed Will one Client/Server protocol subsume all others? –distributed optimization remains an issue Synchronization / Concurrency Control –autonomy of sources negates current concepts –if modules share databases, then database locks may span setup/terminate all for a megaprogram handle. Will software vendors consider moving to a service paradigm? –need CHAIMS demonstration for evaluation

43. June 1998 CHAIMS43 Composition of Processes... versus composition and integration of Data »data-warehouses »wrapping data available on web versus composition of Components »reusing small components via copy/paste or shared libraries locally installed »large distributed components within same “domain” as composition, e.g. within one bank or airline CHAIMS: » processed information » composing autonomous execution threads

44. June 1998 CHAIMS44 Summary CHAIMS requires rethinking of many common assumptions –gain understanding via simple examples Work focused on CALL statement decomposition – to accomplish integration of large services – exploit inherent asynchrony First version of architecture and language drafts are completed; basic infrastructure partially available (compiler, wrapper templates). More demos will come soon. Half-way through a four year project.  http://www-db.stanford.edu/CHAIMS

45. June 1998 CHAIMS45

46. June 1998 CHAIMS46 Other systems (1) Darwin/Regis: »configuration language, architecture definition language, allowing various communication mechanisms (yet all in C++ and under local control) »local (distributed) architecture, non-autonomous, system not only specifies, generates and controls composition, but also basic components »only for homogeneous environment (C++), no legacy Polylith/Polygen: »changes source-files (C, Ada, Pascal, Lisp) according to annotated design data (architecture information) for distribution »only for local distribution (access to source code) and only for TCP/IP

47. June 1998 CHAIMS47 Other systems (2) Hadas: »network centric framework with peer-to-peer communication »Java-based (RMI) with Java-wrappers for non-compliant objects »concept of ambassador objects which are copied to the peer object »allows administrative and design autonomy for objects (heterogeneity, changes of methods) KQML: »protocol for asynchronous communication among agents »allows explicitly different ontologies »offers primitives for getting and updating data, but also for advertising available knowledge

48. June 1998 CHAIMS48 Other systems (3) Differences: »autonomous megamodules, legacy modules »higher-level (just invoke and extract, no specification of various kinds of communication; no specification of module content) »heavy-weight services: ESTIMATE, parallelism by automatic scheduling of invocations according to run-time estimates and cost-functions, EXAMINE, SETPARAM »heterogeneity in distribution protocols focus so far: heterogeneity in distribution protocol on client side focus in future: using ESTIMATE and EXAMINE, automatic optimization and invocation scheduling

49. June 1998 CHAIMS49 CHAIMS proves that... We can do composition in a high-level language. »same language for Java-RMI-invocations and CORBA- invocations (and DCE, DCOM, TCP/IP protocols) »(single megaprogram can deal with multiple protocols simultaniously) »multiple megamodules can run in parallel Large-scale composition can be automated. »in contrast to manual non-software composition (e.g. telephone, cut&paste) »in contrast to fixed programs for one specific problem (e.g. transporting military goods within US) We can do schedulings of programs in a way right now only smart logistics officers can do, avoiding unnecessary waits. »Scheduling of invocations can be optimized.

50. June 1998 CHAIMS50 Long-term Objectives of CHAIMS 1 Implementing a system for a simple and purely compositional language hiding differences of diverse protocols 2Automatic optimized scheduling of invocations (taking advantage of inherent parallelism and estimate- capabilities of megamodules, hence splitting up of CALL-statement) 3Decision-making support (direct) interaction with megamodules, based on overview and incremental results (fixed flow, not yet interactive changes to megaprogram) 4Automatic dataflow optimization (direct dataflows between megamodules), not yet

51. June 1998 CHAIMS51 Assumptions, Additional Constraints Heterogenous legacy modules ==> wrapping of modules, mixing protocols on client side or in wrappers. Parallelism of megamodule-methods not through multithreading on client side but through splitting up CALL-statement (==> sequential program on client side); this leads to useful parallelism because we deal with coarse-grain parallelism. CHAIMS-compliancy for megamodules is achieved by wrapper- templates, for new native megamodules as well as for legacy ones (CHAIMS-compliancy is more than just knowing CHAIMS-protocol!). No reliance on existence of one specific higher level protocol like CORBA, DCOM, RMI ==> implementing an independent data-encoding and marshalling with ASN.1, instead of using one of them and then having converters in the wrappers. Interfaces of megamodules match no investigation into opaque datablobs on client side necessary. Thin client, client should be able to run anywhere (not quite fulfilled right now - we need local ORB, DCE, JavaVirtual-machine). Clear seperation client - server, minimal repository.

52. June 1998 CHAIMS52 Non- (not yet)-Objectives of CHAIMS No commercial product. No specific controls over ilities (security, name- serving, etc.) that they are normally present in distributed systems. No sophisticated front-end, no graphical programming/composition, no browser for repository, no higher-level language as input (not yet). Not solving all problems of megamodule composition that are mentioned in the various CHAIMS-papers (e.g. differing ontologies, non-matching interfaces of megamodules), only the ones mentioned in objectives and additional conditions.

53. June 1998 CHAIMS53 Short-term Objectives of CHAIMS Rest of 1998: »Basic infrastructure (fixing most severe flaws, moving to consistent architecture, all primitives, types, associative lists with handling it, having CORBA) ==> conceptual and implementation work -- CONSOLIDATION »More examples (descriptions of scenarios as well as implemented demos), wrapping one (maybe two) additional suites of megamodules. ==> implementation work -- CONSOLIDATION »Mixing of protocols in client (CORBA, RMI) or/and TCP/IP-three-tier architecture »Preparing for more capable scheduler (examples with current scheduler, reading about other scheduler-problems and implementations, redesigning architecture of compiler (interpreter?), designing scheduler algorithm and architecture, writing paper about all this…) ==> lots of conceptual work, some implementation -- looking ahead for better Scheduler 1999 (depending on where we are at the end of 1998): »Scheduler

54. June 1998 CHAIMS54 Upcoming Changes to Architecture: Other Approach to Heterogeneity Client (megaprogram) native server 1 native server 3 native server 2 chaims compliant module chaims I/O module RMI wrapper TCP/IP sockets CHAIMS protocol CORBARMI server-specific protocols sites of servers client site different wrapper site RMI wrapper CORBA wrapper

55. June 1998 CHAIMS55 Reasons Alternative Architecture Overall: Simpler architecture: less wrappers, just one protocol on client side Server-side: No direct linking with legacy code also for CORBA-wrappers, different sites for wrapper and legacy megamodule possible All native CHAIMS-megamodules will be built using wrapper templates ==> no reason for several protocols, they can all use TCP/IP. Dataflow-optimization: direct messages between megamodules/their wrappers necessary (without bridges) Client-side: Thin client that could run everywhere (TCP/IP is available everywhere, but not CORBA or DCE, RMI also is easily available everywhere). CSRT could be implemented by interpreter instead of compiler, maybe also possible with current architecture, but more complex. We use just transport-facility (really true? what about native CHAIMS-types like string, integer, boolean?) of CORBA, RMI, DCE (for data we have ASN.1); this is already offered by TCP/IP ==> no unnecessary overkill Drawback: missing one of the current funding objectives (heterogeinity on client side).

56. June 1998 CHAIMS56 Upcoming Changes to Architecture: Interpreter instead of Compiler megaprogram in CHAIMS language client code in C, C++, Java, stub code executable client (CSRT) user Compiler: CHAIMS execution machine (interpreter and scheduler) user Interpreter: network CHAIMS-protocol complete megaprogram in CHAIMS language some CHAIMS statements user serves as input to serve as input to CHAIMS compiler, simple scheduler Idl-file generator and compiler C++, Java compiler and linker network

57. June 1998 CHAIMS57 Interpreter Dynamic scheduler: »Input is parsed and stored in a dependancy graph. »Execution machine (interpreter / scheduler) works through the graph and makes appropriate calls: –estimate-calls are added in order to get necessary run-time information for scheduling (cost-function) –every invocation is issued as soon as possible (data-flow) and reasonable (according to cost-function) –all invocations for which the CSRT waits for results are polled regularly, and results extracted and new invocations issued as soon as possible CSRT would still be sequential! Overview results, flexible interactions: »megaprogrammer can program statement by statement and get results immediately; results will influence what he/she does next »like ftp, web

58. June 1998 CHAIMS58 Example: parameters of method xz general default values client-specific settings for client A method xz, client A: invocation-specific settings client-specific settings for client B fromdest start- date end- date number 1JAN1998 6OCT19984OCT1998 7OCT19984OCT1998 LASBWI1 2 actual values used in invocation xz1 SJOZRH SFO 1 1 SJO6OCT19984OCT1998ZRH actual values used in invocation xz2 FRA 7OCT1998 9DEC19981DEC1998 9DEC19981DEC19982LASFRA actual values used in invocation xz1 method xz, client A: invocation-specific settings method xz, client B: invocation-specific settings Hierarchical Setting of Parameters

59. June 1998 CHAIMS59 Using the General I/O-Megamodule for Input Corresponding code in CLAM: ex = SETUP("Example") io = SETUP("io") // io is the general I/O megamodule availabe in CHAIMS (myadress = adress) = ex.GETPARAM() io.INVOKE("ask", value = adress,text = "Please enter adress:") ih1 = io.INVOKE("ask", value = myadress, text = "Please enter adress:") (myadress = result) = ih1.EXTRACTWHENDONE() ih2 = ex.INVOKE("method", adress = myadress) ….. Megaprogram General I/0 Megamodule Megamodule Example 2. Invoke method "ask" with default values and type information 3. Extract user-input 4. Invoke desired method with user-input 1. Get default type and value for desired parameter

60. June 1998 CHAIMS60 Using the General I/O-Megamodule for Output Corresponding code in CLAM: …………... // Setups, Invocations, ih1 is the handle of a // method of megamodule examples (myadress = adress_result) = ih1.EXTRACTWHENDONE() ih2 = io.INVOKE("write", value = myadress, text ="Currently the adress is:") Every element of the data structure in parameter adress_result has besides the actual value also type information and descriptive name. Therefore, the general I/O-megamodule can display the parameter in a decent format without any further formatting information. Megaprogram General I/0 Megamodule Megamodule Example 2. Invoke method "write" with result to be displayed 1. Extract result to be displayed

61. June 1998 CHAIMS61 Primitives in PAM Pre-invocation: SETUP: set up the connection to a megamodule SET-, GETPARAM: preset parameters ESTIMATE: get cost estimates for a specific method Invocation and result gathering: INVOKE: start a specific method EXAMINE: test status of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation TERMINATEALL: terminate the connection to a megamodule

62. June 1998 CHAIMS62 Name of Person Gentype A Gentype is a triple of descriptive name, type and value, where value is either a simple type or a list of other gentypes (i.e. a complex type). Possible simple types: given by ASN.1-library and our own conversion routines (integer, real, string, boolean, date-time, sequence of bytes) Example: Person_Information complex First Name string Joe Last Name string Smith Personal Data complex Address Date of Birthdate 13JUN1934 Soc.Sec.No string 345-34-345

63. June 1998 CHAIMS63 INVOKE in various protocols RMI: public long INVOKE(long clientID, String methodName, ChaimsAttrValContainer cont) throws RemoteException; CORBA: void INVOKE ( in CHAIMSCLIENTID clientid,in CHAIMSMETHODNAME methodname, in CHAIMSATTRVALSEQ attrvalseq, out CHAIMSCALLID callid ); DCE:... local C++: long Invoke (char* methodName, AttrValList * plist) local Java: public long INVOKE (String methodName, ChaimsAttrValContainer cont); AttrValList, CHAIMSATTRVALSEQ and ChaimsAttrValContainer are sequences of name-value pairs, adapted to the various languages and systems. INVOKE for local protocols does not need a client ID because for each client there is one instance of the local megamodule.

64. June 1998 CHAIMS64 Setting up connection in various protocols (client code RMI, Java) RMI: System.setSecurityManager(new RMISecurityManager()); ChaimsCompliantObject remote; try{remote=(ChaimsCompliantObject)Naming.lookup(url+mmname);} catch (Exception e){} try {remote.SETUP(clientID);} catch (Exception e){} local Java: Class cl; //Class to be loaded, for which an object will be instantiated try {cl = Class.forName(full_class_name);} catch (Exception e) {} ChaimsCompliantLocal local; try {local = cl.newInstance();} catch (Exception e) {} local.SETUP();

65. June 1998 CHAIMS65 Setting up connection in various protocols (client code CORBA) CORBA (Orbix, C++ client): nameMM_var _pObj0; try {_pObj0=nameMM::_bind(":mmname","site");} catch (...) {} CHAIMSMMID mmid = _pObj0; pObj0->SETUP(mmid); CORBA (Omnibroker, C++ client): CORBA_ORB_var orb = CORBA_ORB_init(argc,argv); // first get reference from a file and then convert it to an object const char* refFile_pObj0 = "io.ref"; ifstream in_pObj0; in_pObj0.open(refFile_pObj0); // error handling omitted here char stre_pObj0[1000]; in_pObj0 >> stre_pObj0; CORBA_Object_var obj_pObj0 = orb -> string_to_object(stre_pObj0); assert(!CORBA_is_nil(obj_pObj0)); // bind now to megamodule io_var _pObj0 = io::_narrow(obj_pObj0); assert(!CORBA_is_nil(_pObj0)); CHAIMSMMID mmid = _pObj0; pObj0->SETUP(mmid);

66. June 1998 CHAIMS66 Alternative Architecture: Other Approach to Heterogeneity Client native server 1 native server 3 native server 2 chaims compliant module chaims I/O module RMI wrapper e.g. TCP/IP sockets PAM CORBARMI server-specific protocols sites of servers client site different wrapper site RMI wrapper CORBA wrapper

67. June 1998 CHAIMS67 Current Architecture: Several Implementations of PAM Client P A M - protocols CORBA-idlDCE-idlJava-class M e g a m o d u l e s W r a p p e r s

68. June 1998 CHAIMS68 CHAIMS Domain expert (composer) writes megaprogram for composition. CHAIMS automates generation of client for distributed system and optimized scheduling. Megamodule provider provides processed knowledge. CHAIMS Megamodules

69. June 1998 CHAIMS69 Client program Domain expert / Composer Distributed Computation Services IO module Infrastructure: Remote, distributed, parallel computation of a, b, c, d, e IO module Wrapper Legacy Module a b c d e Native CHAIMS Module Control Flow Data Flow MEGA modules

70. June 1998 CHAIMS70 e Composer (Domain Expert) d a b c Distribution System (CORBA, RMI…) CSRT, incl. run-time scheduling Megaprogram (written in CLAM) writes CHAIMS Compiler generates MEGA modules CHAIMS Repository adds information to Megamodule Provider Wrapper Templates wraps non-CHAIMS compliant megamodules information run-time execution composition process CPAM protocol

71. June 1998 CHAIMS71 Exploiting parallelism of megamodule pieces

72. June 1998 CHAIMS72 Primitives of CLAM Pre-invocation: SETUP: set up the connection to a megamodule SET-, GETPARAM: preset parameters in a megamodule ESTIMATE: get cost estimation for optimization Invocation and result gathering: INVOKE: start a specific method EXAMINE: test status and progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation TERMINATEALL: terminate a connection to a megamodule Control: WHILE, IF

73. June 1998 CHAIMS73 Exploit parallelism a d (<a+b) e b c (>a+b) i1 e1 e4 e3 e2 i3 i4 i5 i2 e5 time a d e b c i1 e1 e2 e3 e4 e5 i2 i3 i4 i5 time data dependencies execution of a method non-optimized, or handprogrammed exploring parallelism according to estimates and based on dataflow dependencies invoke a method ie extract results

74. June 1998 CHAIMS74 Dataflow optimization Client program Distributed Computation Services IO module a b c d e Control Flow Data Flow

75. June 1998 CHAIMS75 ChaimsCallRecord callIDclientID paramContmethodIDresContObjectWrapperThreadstatusMethodWrapper ~~~~~~~~~ key ~~~~~~~~~ n-1 ~~~~~~~~ n INVOKE( clientID, methodName,attrValContainer) ChaimsThread LocalObjectWrapper ServerObjectMethodXWrapper createNewCallEntry 1 LocalMethodWrapper start 2 callLocalMethod 3 callMethod 4 methodX 5 -> result 6 setPartialResult 7 setResultContainer 8 setResultDone 9 setStatus( DONE ) 10

76. June 1998 CHAIMS76 SETUP( clientID ) addClientEntry ChaimsClientRecord clientID paramCont (default values) ~ key ~ “PA58” DEFAULT (A, ) (B, ) (Z, ) (B, ) (C, ) (A, ) (B, ) (C, ) (Z, ) written by SETPARAM, read by GETPARAM

77. June 1998 CHAIMS77 What’s the problem? C++ Client Java Client Modules written in C++ or Java Modules written in Java CORBA RMI

78. June 1998 CHAIMS78 What’s the problem? Client RMI CORBA Modules

79. June 1998 CHAIMS79 Possible Solution: JNI C++ MasterClient Java Code CORBA RMI JNI

80. June 1998 CHAIMS80 Possible Solution: JNI Java MasterClient C++ Code CORBA RMI JNI C++ Code JNI

81. June 1998 CHAIMS81 CHAIMS Repository CHAIMS repository data presently an ASCII file. Each line conveys information about a specific megamodule, method, parameter, or “synonym” pair. C - style commenting ( /* … */ ). Basically hierarchical model with certain exceptions.

82. June 1998 CHAIMS82 CHAIMS Repository Model

83. June 1998 CHAIMS83 CHAIMS repository browser: Window Layout #1

84. June 1998 CHAIMS84 CHAIMS repository browser: Window Layout #2