Next Generation Distributed Systems: The and dynamicTAO approach Fabio Kon Department of Computer Science University of São Paulo, Brazil

University of Tromsø 2 Introduction Modern Computing Environments l Hardware diversity: embedded systems, PDAs, laptops, workstations, supercomputers. l Software diversity: different programming languages, component architectures, operating systems. l Mobile computers l Mobile users (different accounts in different systems)

University of Tromsø 3 Highly-Dynamic Environments Frequent changes: 1. Structural changes l HW and SW upgrades, OS patches, protocol updates 2. Dynamic changes l availability of memory, CPU, and network bandwidth; connectivity, physical location

University of Tromsø 4 Goal of Current Research l Facilitate management of dynamic, heterogeneous computing environments for: l Users l System administrators l Developers

University of Tromsø 5 The Approach l “The Network is the Computer”, Sun microsystems. l Network-Centrism : l user profiles, user environments l services, applications, components l WYNIWYG: (What You Need Is What You Get) l dynamic instantiation of applications and services l automatic configuration

University of Tromsø 6 From where can we start? l Run on multiple hardware platforms l Run on top of different OSes l Support different programming languages l Support dynamism, late binding, components l Solution: l OMG IDL l CORBA ORBs l Standard CORBA Services (Naming, Trading, Persistence)

University of Tromsø 7 But There Was A Problem l Conventional ORBs were static: l Fixed threading model l Fixed transport protocol: IIOP (over TCP/IP) l Fixed security strategy l Fixed scheduling l Inadequate for a wide range of applications: l Multimedia l Mobile Computing l Adaptive Applications

University of Tromsø 8 Reflective ORB l Allows inspection and dynamic reconfiguration of the ORB internal engine. 1. dynamicTAO : an extension of the TAO ORB [Schmidt] l very complete, big 2. LegORB (now, UIC ) : a component-based ORB l not complete, but expanding l very small (minimal client 6K or 20K, minimal server 30K) 3. OpenORB : [Blair et al], University Of Lancaster l prototype in Python, implementation in C++/COM

University of Tromsø 9 What is missing? l We have: l Reflective Middleware layer supporting distributed objects in a dynamically configurable way. l Standard services for Naming, Trading, Security, Persistence, Transactions, Events. l We still need: l Support for automatic configuration. l Dynamic instantiation of user environments. l Dynamic resource management.

University of Tromsø 10 2K Services l Component Repository l Automatic Configuration l Distributed Resource Management l Mobile Configuration Agents l User Environment Service l Distributed QoS Compilation Service l Security, Data Management,...

University of Tromsø 11 The 2K Architecture

University of Tromsø 12 Automatic Configuration Service l Automatically instantiates applications and services by assembling their components. l Based on l Prerequisites : static representation of dependencies. l ComponentConfigurators : dynamic representation of dependencies.

University of Tromsø 13 Prerequisites l What a component needs to run: l nature of hardware resources l share of the hardware resources l software services (i.e., components) it requires l Video Client example: l PC with Sound card l 50% of CPU >300MHz l CORBA Video Service

University of Tromsø 14 Automatic Configuration Process 1. Fetches component code and prerequisites from the Component Repository. 2. Dynamically link component code into the application address-space. 3. Based on the prerequisites, repeats the process for other components.

University of Tromsø 15 Automatic Configuration Architectural Framework Component Repository Prerequisite Parser Prerequisite Resolver QoS-Aware Resource Manager Cache load application return reference fetch prerequisites fetch components

University of Tromsø 16 Component Configurators l Reify dynamic inter-component dependencies. l Created on-the-fly by the Prerequisite Resolver. l System and application software can inspect and reconfigure the Dependence Graph.

University of Tromsø 17 ComponentConfigurator Framework l Allows browsing, inspection, and reconfiguration l Can be customized through inheritance l Clear separation of concerns

University of Tromsø 18 QoS-Aware Distributed Resource Management l Global Resource Manager (GRM) l one in each cluster l maintains an approximate view of the cluster resource utilization l Local Resource Manager (LRM) l runs in each node l exports the state of the local resources l Has a Real-Time Scheduler (DSRT) l admission control, reservation, and scheduling

University of Tromsø 19 Loading an Application with the Resource Management Service 1. Client contacts local LRM, giving application name and QoS requirements 2. LRM performs admission test 3. Request forwarded to GRM 4. GRM forwards request to best candidate 5. Remote LRM performs admission test, reservation, and runs AutoConfig.

University of Tromsø 20 AutoConfig Service Loading Several Components

University of Tromsø 21 The 2K Architecture

University of Tromsø 22 Dynamically Configurable Middleware: Reflective ORBs l Reflective Systems [Smith 84] l Meta-Object Protocol [Kiczales 91] l Reflective ORBs [Singhai and Campbell 97] l The ORB maintains a representation of its own internal structure, supporting l Inspection l Dynamic Reconfiguration l Causal Connection

University of Tromsø 23 dynamicTAO l Built as an extension of the TAO ORB [Schmidt et al] l Written in C++ l Modular design based on object-oriented design patterns l TAO already supported startup configuration; configuration file specifies strategies for l concurrency (threading model) l request demultiplexing l scheduling l connection management

University of Tromsø 24 Adding Support for Dynamic Configuration dynamicTAO exports an interface called DynamicConfigurator, supporting 1. Transfer of components across the distributed system 2. Loading and unloading components 3. Inspecting and modifying the configuration of the ORB (and of applications running on top it)

University of Tromsø 25 dynamicTAO Architecture

University of Tromsø 26 Reifying the ORB Structure ComponentConfigurator framework l Stores inter-component dependencies l Allows browsing, inspection, and reconfiguration l Can be customized through inheritance

University of Tromsø 27 dynamicTAO Structure

University of Tromsø 28 DynamicConfigurator IDL Interface interface DynamicConfigurator { stringList list_categories (); stringList list_implementations (in string categoryName); stringList list_loaded_implementations () stringList list_hooks (in string componentName); string get_hooked_comp (in string componentName, in string hookName); string get_comp_info (in string componentName);.

University of Tromsø 29 Manage Component Implementations loaded in memory long load_implementation (in string categoryName, in string impName, in string params,...); void hook_implementation (in string loadedImpName, in string componentName, in string hookName); void suspend_implementation (in string loadedImpName); void resume_implementation (in string loadedImpName); void remove_implementation (in string loadedImpName); void configure_implementation (in string loadedImpName, in string message);

University of Tromsø 30 Manage the ORB Persistent Component Repository void upload_implementation (in string categoryName, in string impName, in implCode binCode); void download_implementation (in string categoryName, inout string impName, out implCode binCode); void delete_implementation (in string categoryName, in string impName); };

University of Tromsø 31 Example of Dynamic Configuration 1. myRemoteOrb->upload_implementation (“Security”, “superSAFE”, superSAFE_impl); 2. newSecurityStrategy = myRemoteOrb->load_implementation (“Security”, “superSAFE”); 3. oldSecurityStrategy = myRemoteOrb->get_hooked_comp (“dynamicTAO”, “Security_Strategy”); 4. myRemoteOrb->hook_implementation (newSecurityStrategy, “dynamicTAO”, “Security_Strategy”); 5. myRemoteOrb->remove_implementation (oldSecurityStrategy);

University of Tromsø 32 Consistency l Dynamic reconfiguration may break the consistency of the internal ORB engine. l Consistency must be ensured by the ORB developer and by the component developer. Achieved by creating customized subclasses of the ComponentConfigurator class: l TAOConfigurator l Servant1Configurator l MonitoringStrategyConfigurator l...

University of Tromsø 33 Implementing Reconfigurable ORB Components l Two major things to consider: 1. Transferring the state from the old component to the new component 2. Making sure that no dangling references to the old component remain Must customize TAOConfigurator or strategy configurator ( e.g. ThreadPoolConfigurator )

University of Tromsø 34 Accessing the ORB Reconfiguration Interface 1. Local or remote code through IDL 2. Telnet 3. Java GUI 4. Reconfiguration Agents

University of Tromsø 35 DOCTOR D ynamic O RB C onfiguration T ool

University of Tromsø 36 Mobile Agents l A mobile agent visits a collection of ORBs. l In each ORB along its path, it can l install new components on the disk, l dynamically link new components, l inspect the state and configuration of the ORB and the applications on top of it, l reconfigure ORBs and applications.

University of Tromsø 37 A Flexible Framework l Different NetworkBrokers support different agent flavors. For example: l simple, lightweight, script-based agents (carrying data and DCP commands only). l powerful, heavyweight, Java-based agents (carrying data, bytecode, and dynamic state, taking autonomous decisions). l Simple agents are suitable for PDAs, embedded systems.

University of Tromsø 38 Reconfiguration with Mobile Agents l SysAdmins use a GUI to build agents for l reconfiguration l inspection l GUI is used to 1. Build distribution graph 2. Select reconfiguration and inspection commands 3. Visualize results.

University of Tromsø 39 Security l SecureAgentBroker uses the GSS-API and supports Role-Based Access Control. l Agents are signed and transmitted via secure connections, using encryption. l RBAC is used in each ORB to decide which commands each agent is allowed to perform.

University of Tromsø 40 The SecureAgentBroker

University of Tromsø 41 Open Problems l Support for fault-tolerance: l fault-recovery when part of the reconfiguration process fails within a node l fault-recovery when the reconfiguration fails in part of the distributed system l atomic transactions l Deploying agents for (re)configuration of active spaces in ubiquitous computing.

University of Tromsø 42 Applications of Reflective ORBs l Completed Prototypes: l Flexible Object Monitoring Service l Dynamic Security Service l Multimedia applications (Nahrstedt, U. Illinois) l Ongoing work: l Ubiquitous Computing (Illinois) l Framework for Adaptive Applications (U. São Paulo)

University of Tromsø 43 Monitoring Distributed Object Interactions l dynamicTAO shows how to adapt l Applications also need to know when to adapt l Monitoring Service: l Can be dynamically loaded and unloaded l No modifications in the applications l Totally transparent to applications l Uses the CORBA request-level interceptor [OMG98a]

University of Tromsø 44 Monitoring Service Architecture

University of Tromsø 45 Monitoring Service Overhead l String getHello (); l Overhead: when monitoring getHello : 10.1% with Monitoring Service on, but without monitoring getHello : 2.0% Ultra-2 Ultra-60 ClientServer Fast Ethernet

University of Tromsø 46 Dynamic Security Service Prototype l Can be dynamically loaded and unloaded l Uses l CORBA interceptors for access control l Cherubim Security Framework [Campbell & Qian 98] l Java Active Capabilities flexible dynamic policies l implemented: DAC, MAC l working on: RBAC, ABAC (?)

University of Tromsø 47 Open Problems l Improving Security Services l how to provide security for millions of distributed objects efficiently? l Monitoring Service tools: l Specify what should be monitored l Visualize monitored data graphically

University of Tromsø 48 The Future l As computing devices become pervasive in our society, we will encounter l highly dynamic, heterogeneous environments l complex dependencies l difficult management l We need standards and an integrated architecture to help manage this complexity in a clean and efficient way.

University of Tromsø 49 End of Part 1 l Questions?

University of Tromsø 50 Security Architecture l Java Active Capabilities l Flexible Security Policies l Caching of Authorization Decisions l Auditing

University of Tromsø 51 Example of Consistent Dynamic Reconfiguration l Concurrency strategies 1. Reactive (single-threaded) 2. Thread-per-Connection 3. Thread-Pool l Switching from 1 or 2 to any other: OK l Switching from Thread-Pool: problematic