RepoMan A Component Repository Manager for Enterprise DRE Systems Stoyan Paunov & Douglas C. Schmidt Vanderbilt University Institute for Software Integrated Systems (ISIS)

Context: Service–Oriented Architectures Historically, distributed real-time & embedded (DRE) systems were built directly atop OS & protocols Operating System & Communication Protocols Hardware Devices Applications

Traditional methods of development have been replaced by middleware layers to reuse architectures & code for enterprise DRE systems Viewed externally as Service-Oriented Architecture (SOA) Middleware Operating System & Communication Protocols Hardware Devices Domain-Specific Services Common Services Distribution Middleware Infrastructure Middleware Historically, distributed real-time & embedded (DRE) systems were built directly atop OS & protocols Service-Oriented Architecture Middleware Context: Service–Oriented Architectures

Applications Traditional methods of development have been replaced by middleware layers to reuse architectures & code for enterprise DRE systems Viewed externally as Service-Oriented Architecture (SOA) Middleware Operating System & Communication Protocols Hardware Devices Domain-Specific Services Common Services Distribution Middleware Infrastructure Middleware Historically, distributed real-time & embedded (DRE) systems were built directly atop OS & protocols Multi-layer Resource Manager (MLRM) e.g., DARPA Adaptive & Reflective Management System (ARMS) program’s Multi-layer Resource Manager (MLRM) MLRM leverages standards-based SOA middleware to manage resources for shipboard computing environments Context: Service–Oriented Architectures dtsn.darpa.mil/ixodarpatech/ixo_FeatureDetail.asp?id=6

Applications Domain-Specific Services ARMS Multi–Layer Resource Manager (MLRM) ARMS MLRM architecture includes Top domain layer containing components that interact with the ship mission manager Middle resource pool layer is an abstraction for a set of computer nodes managed by a pool manager Bottom resource layer manages the actual resource computing components, i.e., CPUs, memory, networks, etc.

Problem: New Complexities Applications composed of loosely- coupled, course-grained distributed components Large number of deployment & configuration artifacts Need to shield deployment & configuration logic from heterogeneous hardware/software environments Continuous system evolution Enable the dynamic redeployment & reconfiguration of application & middleware components Need to provide the right implementation under the right circumstances Applications Operating System & Communication Protocols Hardware Devices Domain-Specific Services Common Services Distribution Middleware Infrastructure Middleware Multi-layer Resource Manager (MLRM)

Solution: Standardized Deployment & Configuration (D&C) Need an administrative entity which RepoMan: an implementation of OMG CCM Repository Manager specification tailored to the needs of enterprise DRE systems Component A v1 … Component B v1 … Facilitates online upgrades, reconfiguration, & redeployment of components Component B v2 … Keeps track of software implementation artifacts & their configuration metadata in heterogeneous environments

Overview of RepoMan Organizes various configurations of component packages Knows how to parse the XML metadata Keeps track of component interfaces & their relationships Facilitates the dynamic update of components at run-time Component A v1 … Component B v1 … Metadata XML Descriptors Implementation Artifacts bundled via ZIP compression have *.CPK extension CORBA HTTP Filesystem REPOMANREPOMAN

RepoMan Design Challenges The CCM specification defines the interface, but it does not prescribe any design & implementation details We were faced with a number of challenges which we discuss next CORBA Object HTTP Server File System Component … Client ADMINADMIN Efficiency Concurrency Synchronization Consistency Portability

Challenge 1: RepoMan & HTTP Server Architecture Context The RepoMan architecture consists of a CORBA object & an HTTP Server Need to implement the CORBA interface specified by the CCM Specification Need to allow retrieval of implementation artifacts via HTTP Problem Shared memory Lack of portability Inflexible Extend the CORBA server & client Limited to CORBA Unnecessary effort HTTP Server CORBA Object File System The Triangle of Mystery

Solution: Loose Coupling Between CORBA Object & HTTP Server No explicit knowledge of each other Do not share internal state Share a common filesystem JAWS vs. Apache The CORBA object supports the standard set of operations enabling manipulation & querying of data in repository retrieval & updating of configuration information A collocated HTTP server enables retrieval of implementation artifacts I = install C = configure Q = query R = retrieve D = delete LEGENDLEGEND CORBA Object HTTP Server Clients I,C,Q,D R ADMINADMIN File System

Challenge 2: PackageConfiguration Complexity Context PackageConfiguration – one of the most complex elements in CCM Location of implementation artifacts not known at package creation Problem Monolithic components 8 levels of hierarchy Assembly-based components 11 levels of hierarchy Programming nightmare Need to update the locations of the implementation artifacts in the PackageConfiguration

Solution: Use Visitor pattern to Cope with Accidental complexity of PackageConfiguration Intent centralize operations on an object structure so that they can vary independently but still behave polymorphically Applicability class relationships of objects in the structure rarely change, but the operations on them change often algorithms keep state that’s updated during traversal Structure Logic for handling each element has a well defined scope

Challenge 3: Cost of Data Movement & XML Parsing Context Manipulating contents of component packages Problem Data movement costs: Uncompressing Writing to disk Parsing with XERCES-C Storing in equivalent C++ classes XML parsing PackageConfiguration parsing involves on the order of tens of files & is therefore very slow Un-ZIP Disk Parse XML C++ data Structure Read Package uncompress Read descriptors Populate Retrieve Store Local HTTP

PackageConfiguration Solution: Minimizing Data Movement & XML Parsing to Improve CPU & I/O Usage File System CORBA Object *.cpk */descriptors/* */implementations/* Extract the package contents only once at installation Wasteful on hard disk usage, but avoiding a bulk of CPU & I/O processing *.EPC Externalizing via CORBA CDR to minimize XML parsing Leveraging the cost of externalizing over lifetime of PackageConfiguration Humanly visible speed-up of metadata retrieval operations myPackage.cpk Install

Challenge 4: Organizing & Managing Data Efficiently Context Need to store local copies of packages Problem Ensure consistency Access, traversal & clean-up No standard filesystem API Need for portability precludes a tightly coupled approach Third-party libraries place limitations on the user community File System Component … … InstallDelete

Solution: Use Package Name & Structure Use the fact that installation name uniqueness in enforces to device the local directory structure Mimic the package structure on the local filesystem Use layout of package to guide you through the clean-up process File System myPackage.cpk myPackage.epc myPackage/descriptors/* myPackage/implementations/* myPackage.cpk Install

Challenge 5: Scalability & Synchronization Context Need to provide high scalability & efficiency Need to preserve consistency Concurrency & synchronization Problems Concurrency strategy choice Single-threaded model Thread-per-request model Can exhaust system resources & degrade performance Thread pool Synchronization inherently incurs overhead Thread-per-Request Thread Pool CORBA Object Client CORBA Object

Solution: Use a Synchronized Variable-size Threadpool to Handle Incoming Requests Scalability Thread pool implemented with the Leader-Followers pattern to handle requests Hash maps to keep internal state O(1) best case access Synchronization Synchronize only the hash maps, not the entire functions Much more efficient than the Monitor pattern

CORBA Object HTTP Server File System Component … Client ADMINADMIN Lessons Learned Building enterprise DRE systems requires a component repository to keep track of software implementations & their configuration metadata Thereby enabling the automated (re)deployment & (re)configuration of the system Applying patterns helps ensure good design Patterns applied to RepoMan included Visitor, Null Object, Leader-Followers, & Memento in the CORBA object Strategy, Bridge, Service Configurator, Iterator, & Singleton in HTTP server Amortizing cost over the lifetime helps improving performance

CORBA Object HTTP Server File System Component … Client ADMINADMIN Concluding Remarks RepoMan strikes an effective balance between flexibility and efficiency. It keeps client code simple and supports system (re)deployment and (re)configuration and dynamic updates. RepoMan is available as part of the Component Integrated ACE ORB (CIAO) at For more information on JAWS check:

Any Questions

Functionality of RepoMan Installation installPackage () createPackage () Deletion deletePackage () Retrieval findPackageByName () findPackageByUUID () Querying getAllNames () getAllTypes () findNamesByType () Retrieving implementations Clients Package PC = PackageConfiguration IA = Implementation Artifact CI = Component Interface IN = installation name HDD CORBA Object HTTP Server 1.Validate Package content 2.Install IA 3.Associate package w/ IN 4.Associate CI w/ IN HTTP

Functionality of RepoMan Installation installPackage () createPackage () Deletion deletePackage () Retrieval findPackageByName () findPackageByUUID () Querying getAllNames () getAllTypes () findNamesByType () Retrieving implementations Clients Find IN PC = PackageConfiguration IA = Implementation Artifact CI = Component Interface IN = installation name HDD CORBA Object HTTP Server Return PC

Functionality of RepoMan Installation installPackage () createPackage () Deletion deletePackage () Retrieval findPackageByName () findPackageByUUID () Querying getAllNames () getAllTypes () findNamesByType () Retrieving implementations Clients PC = PackageConfiguration IA = Implementation Artifact CI = Component Interface IN = installation name HDD CORBA Object HTTP Server IA.dll