1. LIME II LIME II Reengineering a Mobile Middleware

2. A New TupleWare.. Reengineering LIME – a middleware based on the tuple space paradigm revisited according to mobility Goals of the reengineering process weaken the semantics of the model so that it can deal with highly dynamic scenarios decouple the middleware from the underlying application-level transport layer Background: tuple space and LIME v.1

3. Challenges of Mobility Problems the environment changes rapidly and continuously clients connect to the network for short periods of time disconnection is unannounced and unpredictable... How can we manage the increase of complexity ? use more sophisticated SW logics ( ↓ ad-hoc solution) add a middleware layer ( ↑ S.E. solution) Traditional RPC-based middleware is too limited

4. Mobile Middleware A new area of research none.NET Compact Framework, J2ME revised traditional LW-IOP, Mobile DCE, JMS QoS oriented Mobiware, L 2 imbo reflection based Universally Interoperable Core mobile code μCode, Aglets, Mole context aware Nexus, Teleporting data-sharing oriented Coda, Bayou, Odyssey coordination based reconfigurable pub/sub tuple spaces Commercial success of tuple space based technology: TSpaces (IBM) and JavaSpaces (Sun)

5. Tuple Spaces Original model developed in the mid-80’s (Linda) Space of tuples astraction, a separate third-party form of distributed shared memory that is accessed concurrently by processes that insert, take or read tuples Tuple a “generic data structure”, a sequence of typed fields a tuple field is either an actual or a formal tuples that have formal fields are also called templates simple matching rules e.g. ≈

6. Primitives out(tuple) inserts the tuple into the tuple space rd(template) reads a tuple that matches the given template, blocking in(template) takes a tuple that matches the given template, blocking Extensions non-blocking: rdp, inp bulk: rdg, ing Non deterministic matching out out copy rdp rdp ing ing in in wait ? ?

7. Pros and Cons Temporal decoupling ( ↑ ) asynchronous communication: consumers and producers can exist in ≠ periods of time Spatial decoupling ( ↑ ) no bindings between servers and clients: requests of tuples are implicitly routed decentralized, fault-tolerant architecture Global and persistent tuple space ( ↓ ) unrealistic assumption for mobile scenarios

8. Linda In a Mobile Environment Virtual spaces, name-based “transient sharing” Federated Tuple Space Host-Level Tuple Spaces Interface Tuple Spaces Agent

9. Constructs of LIME Location-aware operations out: select the agent that has to hold the tuple rd, in: select a projection of the tuple space Reactive programming reaction = a code fragment that is executed when the state of the space satisfies a given template strong vs weak They cope with physical and logical mobility

10. Engagement / Disengagement space grows and shrinks transparently according to the connectivity among the components engagement – the sharing process, atomic disengagement – the inverse process, atomic engagement disengage

11. Roles of Engagement Engagement modifies only the accessibility of tuples: each tuple exists in a single location Engagement is used to attach a new node to the system move tuples whose destination is the new node disseminate reactions regarding the new node The engagement process delivers tuples and reactions as soon as the target agent or host joins the LIME community

12. Other Features Applications can retrieve information about the community of agents and hosts by accessing a “system” tuple space (LIME System Tuple Space) Other components: μCode – kernel for mobile agents LighTS – tuple space engine and adapter

13. Limitations of LIME v.1 Engagement relies on distributed transactions to complete the transaction, all hosts have to stay connected for enough time since transactions are atomic, they block all hosts until completion Disconnection must be announced if unannounced disconnection occurs while tuples are transmitted, they are lost the system is not able to recover whenever a host collapses during a transaction

14. My Work: LIME II 1. Reengineering the model 2. Reengineering the implementation 3. Improvements and optimizations

15. Changes to the Model LIME LIME II mobile agents are supportedexplicit upload of code is supported tuple spaces can be bound to agentstuple spaces are bound to hosts group view is symmetricalgroup view is asymmetrical group join and reconciliation of tuples and reactions occur during engagement group join and reconciliation are asynchronous access to the tuple space is globally atomic the semantics for accessing the tuple space is not globally atomic unannounced disconnection is not supported unannounced disconnection is supported and policies for reconciliation are provided

16. Engagement Vs Reconciliation Dis/Engagement allows a host to join or leave the group each host collects the beacons from the other units and updates its (asymmetrical) view of the community Reconciliation allows to deliver misplaced tuples, register issued reactions and remove triggered ones, resolve inconsistencies of the space atomic / wrt a pair of hosts and a single tuple space B B B B

17. Best-Effort Unannounced disconnection can occur while tuples are being transferred to a host clean-up phase of TCP/IP: the sender perceives failure and cannot know whether the data was handed to the receiver Multi-level approach 1. transport re-send the message declare the type of failure: critical, total, partial 2. application protocol apply the recovery policy selected by the user for this space loss, duplicate, conservative the conservative policy keeps track of the tuples whose delivery we are uncertain of  reconciliation will be run (third level)

18. Architecture of LIME II Decoupled with respect to the underlying application level transport layer (reconfig pub/sub, DHT, etc..) applications modules (3 rd party) LIME II communication adapter network Plain Unicast + Multicast Subject based Pub/Sub Content based Pub/Sub... DHT based MW

19. A New Design Decoupling requirement was imposed by VICOM funded Italian research project that embraces LIME Other goals build a modular, flexible, scalable middleware increase overall performance solve non-trivial synchronization issues – that are due to the weakened semantics of the model, such as concurrent reconciliations race conditions on accessing the tuple space Radical reengineering only a small fraction of the code has been reused ≈ 2% substantial compatibility with the API of LIME v.1 underlying architecture is extremely different

20. Core of the Middleware LimeTupleSpace main class, de/limification, operation descriptors LocalOpMgr synchronization of agents, local operations LimeTSMgr astraction of a local reactive tuple space Reactor support for strong reactions, atomicity TupleSpaceProxy wrapper for LighTS, cloning, naming CommunicationMgr remote operations via a thread pool CommunicationAdapter adapter for the transport layer

21. Communication Adapter You can employ any application-level transport layer for whom it is provided a (third-party) adapter that implements the standard interface ICommunicationAdapter init() shutdown() joinSystem(info) leaveSystem() joinGroup(space) leaveGroup(space) sendOp(space, message) sendOp(host, message) sendReaction(space, message) sendReaction(host, message) sendReply(host, message) getNextOp() queryHost(host) addHost(host) remove(host) Available communication adapters LocalTestCA pure UDP unicast, only for testing LocalCA UDP unicast + multicast (wired LAN, high performances) PlainCA TCP unicast + multicast (connection caching, keep alive) MessageCA decorator, shows flow of messages FragProxyCA enables de/fragmentation of messages

22. Remote Operations Thread-to-thread messaging among thread pools of hosts automatic dispatching of msgs according to headers 37 protocols implement all the operations classified on the basis of type, space projection, snd/rcv side instances of Runnable, assigned to the thread pool as “tasks” kept apart from the infrastructure a protocol needs to send and receive messages to send messages, the thread invokes the methods of C-Adapter to retrieve messages, the thread queries a non-shared buffer a timeout mechanism avoids to wait forever on the buffer ProtocolsThreadPool local agent or remote request selectProtocol(...) ùPùP assignTask(P) P : RemOutg CommunicationMgr

23. Synchronization A reaction is always executed atomically, even when its code fragment installs other reactions strong → weak → strong → weak →... The methods of the core classes of the middleware are re-entrant, i.e. they can be invoked also “within” the bodies of reactions Complex case: reconciliation during a reconciliation between two hosts, the arrival of a tuple can trigger a reaction that starts reconciliation with a third host and, recursively, with a fourth,... the distributed transaction is splitted into parallel processes of reconciliation – they can succed or not according to timeouts, interleaving of threads,... if a reconciliation process fails, the system will retry it later

24. Further Improvements Parallelized operations an operation ( outg, installWeak ) dealing with several hosts is splitted into independent tasks; the tasks are assigned to the thread pool Thread pool anti-leakage, safely resizable on-the-fly profiles the maximum number of busy threads enables fine-tuning and load-balancing Priority scheduler (QoS) manages the internal flow of msgs, parametric

25. Also.. Fragmentation and defragmentation transparent to applications can be turned on/off for any communication adapter allows to transfer extremely large objects (e.g. tuples whose order of magnitude is MBytes or more) Integrated diagnostic service lightweight web server generates dynamically a set of HTML pages that describe the contents of the tuple spaces the service is very useful for testing LIME applications: you can inspect their behavior via a browser

27. Testing and SW Development Agents each agent performs a single operation, allowing to select the projection of the tuple space you can combine agents in order to re-create nearly every possible testing scenario LiM an ICQ-clone instant messenger the whole logic of the application consists of 73 lines of code advances features, e.g. off-line delivery of msgs, cost zero

28. Conclusions The model has been weakened it copes with highly dynamic environments (MANETs) The middleware is modular, scalable, flexible performances are superior with respect to LIME v.1 part of the Italian Research Project VICOM also used in TU Delft, Netherlands Future work implement other communication adapters REDS, DHT, bridge communication adapters,... perform a quantitative analysis of performances