1. Navarro, Marquès & Freitag 1 April 2004 On Distributed Systems and CSCL Joan Manuel Marquès - jmarquesp@uoc.edu Universitat Oberta de Catalunya Leandro Navarro Felix Freitag Universitat Politècnica de Catalunya

2. Navarro, Marquès & Freitag 2 April 2004 Table of contents ● Introduction ● Requirements ● Our proposal: LaCOLLA ● Architecture of LaCOLLA ● Virtual synchronism ● Execution of tasks ● Prototype and validation ● Conclusions

3. Navarro, Marquès & Freitag 3 April 2004 Introduction 1. Asynchronous collaborative applications have to deal with: – Interaction nature ● Participants dispersed at Internet; many-to-many collaboration; people participate at different times/places (home, work, notebook);... – Idiosyncrasy of groups ● Flexibility, dynamism, decentralization, autonomous participation, groups exist while members participate and provide resources, few participants,... – Technical and administrative issues: ● Guarantee information availability; interoperability among applications; security (authorization, access rights, firewalls),... participants belong to different organizations or departments with different authorities that impose rules and limitations in order to facilitate administration, internal work and individual use.

4. Navarro, Marquès & Freitag 4 April 2004 Introduction ● Development of applications that take into account those requirements are costly (in time, resources and economically) ● [Solution]: Simplifications ● Centralization, reduction autonomy, dependency on external resources,... ● [Consequence]: – Development of collaborative applications is a hard work. – Applications include minimal collaborative functionalities – Existing collaborative applications focus on a few of those aspects (the key aspects for the application) and neglect other aspects.

5. Navarro, Marquès & Freitag 5 April 2004 Introduction 2. To have groups that are computationally autonomous, flexible and independents...... ability to execute tasks and deploy services using resources belonging to group is required: – Execute task: use computational resources belonging to group to execute tasks. ● Task: application that has an start and an end. – Deploy services: deploy services in resources belonging to group. Group guarantees that services are available. ● Service: application that executes “for ever”.

6. Navarro, Marquès & Freitag 6 April 2004 Requirements ● Oriented to groups ● Internet scale (distributed, dispersed) ● Universal and transparent access ● Decentralization ● Self-organization of the system ● Individual autonomy ● Self-sufficiency ● Allow sharing ● Resources availability (replication) ● Security

7. Navarro, Marquès & Freitag 7 April 2004 Our proposal: LaCOLLA ● a fully decentralized infrastructure for building collaborative applications that: – provides general purpose collaborative functionalities. – Concept of group – Know what is going on in the group (= disseminate information about actions that take place inside the group to all group members). Awareness. – Capacity to share information generated in the group (= access to objects belonging to group at any moment; independently whether the member was connected or disconnected when the object was created). ● Will avoid applications deal with complexities derived from groups/members ● This simplification will help inclusion collaborative aspects into applications. ● To meet flexibility required by groups, LaCOLLA is decentralized, autonomous, self-organized, uses resources provided by group members... – provides groups computational autonomy and independence: ● ability to execute tasks and deploy services using resources belonging to group

8. Navarro, Marquès & Freitag 8 April 2004 Functionalities ● “Immediate” and consistent dissemination of events ● Consistency in the storage of objects (virtually strong consistency) ● Execute tasks (and deploy services) ● Presence (of members and components) ● Location transparency (of members and components) ● Instant messaging ● Management of groups and members ● Not implemented – Security – Disconnected mode

9. Navarro, Marquès & Freitag 9 April 2004 Architecture of LaCOLLA ● Five kind of components: UA, RA, GAPA, TDA and EA. ● Internal mechanisms to coordinate the components. ● Following peer-to-peer paradigm – (decentralized, every peer can be server and client, self-sufficiency, self-organizing, ad hoc connectivity, scalability,...) ● Use pools of resources (computational, storage) belonging (or available) to members of the group. – Resources are: geographically distributed, owned by different members (different authorities), heterogeneous,...

10. Navarro, Marquès & Freitag 10 April 2004 Components Five kind of components: UA (User Agent): ● Represents the user; ● Interacts with applications RA (Repository Agent): ● Persistent storage (objects and events) GAPA (Group Administration and Presence Agent): ● Administration: groups and members ● Users' preferences ● Authentication of members TDA (Task Dispatcher Agent): ● Distributes tasks to executors EA (Executor Agent): ● Executes tasks Members in the peer decide to instantiate any number of components.

11. Navarro, Marquès & Freitag 11 April 2004 Internet Application A Peer LaCOLLA Transport UA Transport UA RAUA GAPA Peer LaCOLLA Transport UA Peer LaCOLLA Transport UA RAUA GAPA Peer LaCOLLA TDA EA Transport UA RA UA EA Peer LaCOLLA GAPA Transport UA Peer LaCOLLA Application B Application B Transport UA TDA UA GAPA Peer LaCOLLA RA Application B Application B Application B Application A

12. Navarro, Marquès & Freitag 12 April 2004 Mechanisms

13. Navarro, Marquès & Freitag 13 April 2004 Virtual synchronism ● Immediate and consistent dissemination of events – All peers know what is happening in the group: ● All connected peers receive events immediately after they occur. ● Not connected members receive events when they connect. – How is achieved? ● Immediate: the component where the event is generated disseminates it. ● Consistent: weak consistency algorithms to guarantee that eventually all connected components will have all events. ● Virtually strong consistency of objects – Immediate and consistent dissemination of events guaranties that all peers know where each object is located. – Storage components store and reproduce objects in an autonomous way. ● They don't need to worry about consistency. The consistency is achieved because all components know the location of all objects.

14. Navarro, Marquès & Freitag 14 April 2004 Execution of tasks ● Organized in a decentralized and dynamic manner – TDAs coordinate among them in a decentralized manner to schedule tasks. – EAs execute tasks – When the task is finished, the result is provide to the user in an asynchronous way. If user is not connected, the result will be provided to him at his next connection. ● Storage (code and data), presence,... provided by LaCOLLA ● Based on ideas used to design JNGI ( ) – JNGI: a decentralized and dynamic framework for large-scale computations for problems that features coarse-grained parallelization. Components of JNGI communicate using JXTA. ● In our case use communication facilities of LaCOLLA.

15. Navarro, Marquès & Freitag 15 April 2004 Prototype and validation ● Architecture validated by simulation: – To prove basic functionality (virtual synchronism) ● Members know what is happening at a time that members perceive as immediate (while the actions occur). ● Members have access to generated objects right after they are created. – Simulation using J-Sim ● [Now:] Prototype: – basic functionality (implemented) – execution task capability (being implemented) – language: java (future: extend to other languages) ● Applications: – chat (implemented) – sharing documents (being implemented) – execute java applications –...

16. Navarro, Marquès & Freitag 16 April 2004 Conclusions ● An infrastructure that provides general purpose collaborative functionalities (groups, awareness and storage) will facilitate the creation of collaborative applications. ● + infrastructure use resources provided by members: self- sufficiency of groups. ● + infrastructure has the ability to execute tasks (and deploy services): groups will gain in autonomy and independence.

17. Navarro, Marquès & Freitag 17 April 2004 Conclusions (II) ● LaCOLLA is our proposal for an autonomous and self- organized infrastructure for collaboration support that only requires resources provided by group members. – Refined in terms of components and mechanisms. – Validated by simulation. – Prototype implemented + some application. ● Future work: – Extend/complete the implementation (deploy services, security,...). – Extend the range of implemented applications (instant messaging, blackboard, collaborative editor,...). – Use applications in real situations in order to improve the infrastructure.

18. Navarro, Marquès & Freitag 18 April 2004 On Distributed Systems and CSCL Joan Manuel Marquès - jmarquesp@uoc.edu Universitat Oberta de Catalunya Leandro Navarro Felix Freitag Universitat Politècnica de Catalunya

19. Navarro, Marquès & Freitag 19 April 2004 On Distributed Systems and CSCL Leandro Navarro Joan Manuel Marquès - jmarquesp@uoc.edu Felix Freitag

20. Navarro, Marquès & Freitag 20 April 2004 Abstractions ● Concept of group – (= operations at group level; operations involve groups) – Hide complexity of group dispersion;... ● Know what is going on in the group. – Provide a means to disseminate information about actions that take place inside the group to all group members in a transparent way (independently of location of members). ● Capacity to share information generated in the group. – Allow access to objects generated inside the group at any moment; independently whether the member was connected or disconnected when the object was created. ● Execute tasks and deploy services – Allow members of a group user computational resources belonging to group.

21. Navarro, Marquès & Freitag 21 April 2004 Abstractions (II). ● Collaborative groups we are interested in are characterized by: – Few members – Dispersed – Multiplicity (= members belong to one or more groups) – Decentralized organization – Members behave autonomously – Self-sufficient – Not all group members are equal – No fix connection point – Not all peers are equal

22. Navarro, Marquès & Freitag 22 April 2004 Abstractions (III). ● How can we know what is going on in the group? – Disseminating events that inform about actions done by group members – This dissemination should be: ● Immediate ● Consistent

23. Navarro, Marquès & Freitag 23 April 2004 Abstractions (IV). ● How should be the storage of information generated inside groups? – Available: persistent, resilient, reproduced – Decentralized and distributed – Detects conflicts in object handling – Objects located near demand – Dynamic – Storage capacity and bandwidth adapted to group needs

24. Navarro, Marquès & Freitag 24 April 2004 Mechanisms (II) ● Behaviors – Push: send to other interested components ● disseminate event; I'm still alive;... – Pull: ask another component ● consistency sessions;... – Autonomous decisions: decide according to local information ● replication of objects; get object; inactive component;... ● Mechanisms implemented using: – Diffusion / application level broadcast – Weak-consistency optimistic protocols – Random selection ● Transport: reliable channel (TCP)

25. Navarro, Marquès & Freitag 25 April 2004 Table of contents ● Introduction ● Foundations ● Contribution 1: architecture ● Contribution 2: virtual synchronism ● Validation ● Conclusions & future work

26. Navarro, Marquès & Freitag 26 April 2004 Prototype ● LaCOLLA architecture behaves in a self-organized manner. ● Virtual synchronism guaranties that group members: – Know what is happening at a time they perceive as immediate (while the actions occur). – Have availability of generated objects right after the object are created. ● All the above is achieved even though members are using their own resources. ● Presence influences self-organization. ● Consistent view is achieved even though all components are not consistent.

27. Navarro, Marquès & Freitag 27 April 2004 Table of contents ● Introduction ● Foundations ● Contribution 1: architecture ● Contribution 2: virtual synchronism ● Validation ● Conclusions & future work

28. Navarro, Marquès & Freitag 28 April 2004 Validation ● By simulation – Using J-Sim ● Provides network simulation until transport layer. ● Scripting language (tcl) to organize simulations. ● Simulator code in Java. ● Generated code requires few changes to be be used in a real infrastructure. – Simulation variables and parameters ● Kind of groups: size and activity level of its members. ● Levels of dynamism: connection, disconnection, mobility and failures. ● Parameters of mechanisms. ● Degree of reproduction (#RA and #GAPA)

29. Navarro, Marquès & Freitag 29 April 2004 Simulation ● Three cases: – “Normal situation”: ● 10 members; low level of dynamism; different degrees of reproduction. – Effect of dynamism. – Effect of scale: ● small groups, larger groups with little dynamism, very dynamic larger groups. ● Methodology: (for each experiment) – Hypothesis – Definition of the experiment – Results – Conclusions

30. Navarro, Marquès & Freitag 30 April 2004 Simulation (II) Phase 1: show tolerance to failures and changes ● Events and objects are generated depending on activity degree. ● Occur failures and changes (connections, disconnections and mobility) depending on degree of dynamism. ● All internal mechanism are operative. Phase 2: ability to recover automatically ● Calculate how much time requires LaCOLLA to be consistent (Consistent: all components have same information about: presence and location information, available objects, events, GAPA belonging to group). ● Only are active internal mechanisms.

31. Navarro, Marquès & Freitag 31 April 2004 Types of analysis used in the validation (influence of dynamism) ● Percentage of experiments that have necessary resources (RA and GAPA) connected to group at the end of first phase – 10 Members – Low dynamism – 10 Members – High dynamism

32. Navarro, Marquès & Freitag 32 April 2004 Types of analysis used in the validation (II) (automatic recovery) ● 10 Members ● Low dynamism

33. Navarro, Marquès & Freitag 33 April 2004 Types of analysis used in the validation ● 10 Members ● Low Dynamism

34. Navarro, Marquès & Freitag 34 April 2004 Validation summary ● Percentile 90Low Dynamism# RA and #GAPA: 10

35. Navarro, Marquès & Freitag 35 April 2004 Conclusions of validation ● LaCOLLA architecture behaves in a self-organized manner. ● Virtual synchronism guaranties that group members: – Know what is happening at a time they perceive as immediate (while the actions occur). – Have availability of generated objects right after the object are created. ● All the above is achieved even though members are using their own resources. ● Presence influences self-organization. ● Consistent view is achieved even though all components are not consistent.

36. Navarro, Marquès & Freitag 36 April 2004 Table of contents ● Introduction ● Foundations ● Contribution 1: architecture ● Contribution 2: virtual synchronism ● Validation ● Conclusions & future work

37. Navarro, Marquès & Freitag 37 April 2004 Conclusions ● An infrastructure that provides asynchronous collaborative functionalities in an autonomous manner, requires that the infrastructure has the ability to self- organize. ● Virtual synchronism contributes to autonomy because it guarantees immediate and consistent availability of awareness information and generated objects to all members and components. ● LaCOLLA is my proposal for an autonomous and self- organized infrastructure for collaboration support that only requires resources provided by group members. – Refined in terms of components and mechanisms. – Validated by simulation.

38. Navarro, Marquès & Freitag 38 April 2004 Future work ● Implementation of a prototype – Prototype and applications [currently in progress] – Implement internal mechanisms: ● Delete objects ● Conflict detection ● Management of groups and members ● Instant messaging ● Disconnected operational mode ● Handling of anomalous situations ● New mechanisms – Extend internal mechanisms: events transformation; security and anonymity. – Decide best location and optimal deployment of objects – Improve download mechanism (download from many locations) – Allow sharing of resources between groups – Decide best algorithm for each mechanism (improve random techniques)