COTS-Based System Architecture The Key to CBSuccess The purpose of this group has been to analyze the architectural issues related to COTS-based systems.

Break-out Group Participants Seacord Medvidovic Willey Brown Kerner Henchman Roshandel Patel Rakic

Architecture Defined (SEI) What is software architecture? There is no standard, universally-accepted definition of the term, for software architecture is a field in its infancy, although its roots run deep in software engineering. While there is no standard definition, there is also no shortage of them. The definition of software architecture is not a standard one. There are different views, particularly since the field is so young. There is also no standard definition of the terms and concepts of the field. This presents some difficulty since people involved in the field some times do not speak the same language.

Garlan and Perry, 1995 The structure of the components of a program/system, their interrelationships, and principles and guidelines governing their design and evolution over time. Garlan and Perry in 1995 defined the architecture of system as the components, their interrelations and rules governing them. This is one of the definitions used in academia and industry but obviously not universally accepted.

Architecture Perry & Wolf Form – allowable system structure Elements Rationale – hides the "ilities" behind the form Perry and Wolf have defined software architecture as form, elements and rationale. This is another good definition.

Types of Architectures Traditional – custom development (clean slate) Legacy driven – documented practices Emergent – COTS-based Predefined – middleware induced Emerging ways of dealing with COTS when you're architecting things:   - can we say things about whether they will work in particular domains - areas it would be valuable to apply more effort

Emergent Architecture Required "ilities" drives system architecture System architecture, when properly defined and applied, ensures system ilities How do we support system ilities (e.g., security) in emergent architectures? Emergent properties of the architecture arise from characteristics of the components used - perhaps because they can be combined in only one way - architecture is driven by components. One could say that this approach is characterized by a poorly understood problem because we don't fully understand the (technical) problem (the components are black boxes). It is not always feasible to analyze the properties of a system which we have very limited information on.

Reference Model System Integrator Internet Browser SRS Query Query Server VisiBroker Component Specification JDK 1.3 Qualified Components Ranked Ensembles Netscape Navigator This is a diagram of a reference model. One can see that there are different sub systems interacting. This is only an example of a “simple” system. Component Repository Integration Rules Component Ensemble Evaluation

Importance/Difficulty Mappings Following slides map importance/difficulty We first plot everyone's responses for each issue Next, we provide rough average or consensus of opinion for each issue Last, we plot consensus of opinion on a single graph The team went through a series of exercises to categorize the discussed issues based on their importance and difficulties.

Research Projects Identification of attributes Attribute classification Representation - Standard, form, eg. XML Discovery - understand the properties of a given component Knowledge-base of integration rules Resolving integration / architectural mismatch A set of research projects were identified through the session. They mainly involved unanswered questions for analyzing the COTS-based systems and techniques which could be used as a standard guide for using COTS systems.

1. Identification of attributes rr jk wb H rs aw Importance M This diagram shows the importance and difficulty levels of identifying the attributes. L L M H Difficulty

2. Classification H wb Importance M rs rr jk aw L L M H Difficulty This diagram shows the importance and difficulty levels of classification problem. L L M H Difficulty

3. Representation rs wb H aw jk Importance M rr L L M H Difficulty This diagram shows the importance and difficulty levels of representation problem. L L M H Difficulty

4. Discovery rr jk H aw rs wb Importance M L L M H Difficulty This diagram shows the importance and difficulty levels of discovery issue. L L M H Difficulty

5. Knowledge base of integration rules aw rs H jk rr Importance M This diagram shows the importance and difficulty levels of knowledge base integration rules L L M H Difficulty

6. Resolve integration mismatch rr rs aw H jk Importance M This diagram shows the importance and difficulty levels of resolving integration mismatch L L M H Difficulty

Architectural styles vs. architectural patterns Predicting system properties from component properties and architectural styles and/or patterns (how/can we?) Style: Pipe & Filter Pattern: Web Browser Web Server Business Logic DBMS One perspective specifies the need to evaluate components in terms of architectural patterns - comparison of properties/trade-off analysis - how to combine components with other component patters - component patterns in COTS-based systems - are architectural (component) patterns different from architectural styles?

aw nm jk H rs Importance M L L M H Difficulty 7. Predicting system properties from component properties and architectural styles and/or patterns aw nm jk H rs Importance M This diagram shows the importance and difficulty levels of predicting system properties from component properties and architectural styles and/or patterns. L L M H Difficulty

Research Projects Compared 6 7 H 4 1 3 5 Importance M 2 This shows the importance and difficulty level of the proposed research projects. This could be good indicator as of what should be of importance to the academic research software architecture community. L L M H Difficulty

Impact on architecture of post-deployment evolution of COTS in a CBS Dealing with system impact over time Value of Architectural Techniques for minimizing Evolutionary Impact Interface standards Wrappers Architectural styles Architectural patterns One issue here is the impact on architecture of post-deployment evolution of COTS in a CBS - characterize/predict/minimize Another possible Issue is the value of wrapper approach to isolating change? and the effectiveness of this approach in a system with a small number of components or a system with a large number of components Another issue is whether there are rules of thumb that can be used to minimize the impact of COTS component change in the post-deployment phase? (e.g., don't require the vendor to provide a service that is not available off-the-shelf) - interface standards - wrapping - architectural style - a/c patterns - planning and predicting post-deployment evolution is difficult in general, but for COTS-based systems it's much more difficult, because there are many independent variables

Forms of Maintenance Perfective Corrective Adaptive COTS Volatility There are 4 types of maintenance activities identified: 1. Perfective 2. Adaptive 3. Corrective 4. and also COTS volatility-driven

Middleware Is middleware special or should it be treated as just another component Is middleware more of an architectural driver than other components? Is middleware a glorified adaptor between components that provide functionality? What is a middleware? Does it really exist as an independent entity? The views on this issue vary. - impact of overlapping functionality

Pre and Post Conditions Useful to know pre and post conditions on a per method basis How precisely can we identify semantics of a function call, pre- and post-conditions? This is a very important issue and can be very useful. One view defines pre and post conditions as the smallest amount of additional component semantics (on a per-API basis) you need in order to be able to use the component One issue is how to discover pre- and post-conditions of a method or service of a COTS component? One solution would be through an incremental process. One perspective is that this is a detailed design issue and may not be relevant at this point, and needs to be dealt with elsewhere.

Architecture and Design Difference between architecture and design in COTS-based systems vs. custom developed systems What are unique architectural concerns with COTS-based systems? We need to be able to distinguish the difference between design and architecture of the COTS based systems. This issue could be different from the one of the custom developed systems. We need to be able to identify the unique architectural concerns when dealing with COTS based systems.

Ran out of Time… Importance of trade-off analysis Adherence to interface standards Overlap of functionality Can we design guidelines for integrating COTS?? These guidelines may be different depending on what architecture is (green-field, legacy, etc.) and also depending on type of COTS (package, component, COTS) The guidelines could be a checklist of questions.It would be the property of overlapping functionality among COTS pieces - need to figure out how to sort out from the packages that functionality that you need? The major issue here is the comparison of properties for trade-off analysis of components that might be used in a system This could be addressed by iteratively increasing and relaxation of constraints depending on the analysis

Backup Slides

COTS Definition Don't own them  Have (possibly) limited control over evolution  No visibility of internals  Into product structure & evolution Sold for profit  There are different characteristics of COTS system. Some of them are: - Customer doesn't own it. - No control over its evolution (maybe some influence) - No visibility into its internals - sold for profit - "off the shelf" - A priori knowledge that new versions will come out - Limited visibility into what's inside the product, and limited visibility into how it's going to evolve - Difficult to find the right COTS package (2000 new products a month)

COTS Definition Multiple, identical, instantiated, and available. Purchased as a unit Embeds architectural assumptions Vendor determined API Some more definitions/characteristics are discussed: - “Black box" visibility – I.e you got an API - Something you can buy as a unit - Comes with its own architectural assumption - Comes with its own API - or vendor determined API

COTS Definition Is completely done May get too much functionality Difficult to anticipate quality May get too much functionality Hierarchy of COTS Packages Components Frameworks Issue - how can you get this info? Allowable ways of discovering the properties (methods will be different for black box, white box, etc.) There are different types of COTS components with different types of properties which influence the type and nature of integration.(package/component/framework)  

Architecture Boundaries Boundary between design & architecture Architecture is System properties? Functionality implementation? System structure? Is the boundary between design and architecture clear? Is there a consensus on what we mean by them? Design and Architecture of different levels of abstractions for specifying system and software properties.

Emergent Architecture "ilities" drive the solution How can we fulfill the "ilities" Want a secure product but COTS products have not have security built in. “Emergent" architecture paradigm brings up architectural issues - how do we know how we're going to combine them? System "ilities" ordinarily drive an architecture - security, performance. The purpose of an architecture is to ensure that the system demonstrates those properties. Is it on a case by case base or can they be identified once and for all?

Issues How do we know how to combine COTS components to create an architecture? [8] How do we satisfy required properties of the architecture using COTS components?[9] What is the difference between architecture and design for a COTS-based system?[10] Modeling of COTS integration [11] When dealing with COTS based system, we are dealing with different issues than when we talk about the architecture of a non COTS-based system. The issue here is how do we satisfy required properties of the architecture using COTS components? There is another issue here to address which the relationship and borders between design and architecture of a COTS-based system. What is the difference in architecting and in designing for a COTS-based system vs. for a non-COTS-based system? what are the unique architecting concerns that need to be applied when developing a COTS-based system?

Issues - 2 How do you identify properties of COTS components that are pertinent to the system architecture?[7/1] How do you objectively characterize properties of COTS components that are pertinent to the system architecture?[7/1] Characterize activities of assembling COTS components?[2] Other issues of concerns are how can we characterize properties of COTS components that are pertinent to an architecture? (objective way of characterizing properties)  How can we characterize activities of assembling COTS components into an architecture?  Is middleware special, or should it be treated as just another architectural component?  Discovering properties of COTS architecture - e.g., throughput, reliability, performance.  Discovering properties that allow it to be combined into architectures

Issues - 3 Is middleware special, or should it be treated as just another component?[3] How do we mitigate architectural mismatch caused by COTS incompatibilities?[5] Issue: Mitigating/resolving architectural mismatch stemming from COTS incompatibilities - set of actions people take to combine components - resolving architecture mismatches importance - very high - difficulty ranges from med/low to med/hi (others pick spots of med and med/hi, all with highest importance - Incorporate a knowledge base of integration rules importance: med-hi, difficulty; med/hi to hi

Issues - 4 Properties[5] How do we discover properties of components? Quantitative Qualitative How do we discover properties of components? Some may not be quantifiable -> reliability Black box/white box/gray box Some of these properties are domain specific Issue - how can you get this information regarding the properties of the component? Allowable ways of discovering the properties (methods will be different for black box, white box, etc.)

Issues - 5 Representation of properties such as API's[6] PRE-POST conditions[7] How do we discover these in an incompletely specified API Semantics QoS Standard UI Issue: semantics of a function call, pre- and post-conditions. This is a per API issue. And could be viewed as a detailed design issue. However it can be addressed at different levels of granularity.  ("Hidden API" - Incompletely specified API) One problem is that the properties of the component that are critical to integration but are not obvious. How to discover pre- and post-conditions of a method or service of a COTS component? Incremental process could be a way to a achieve these goals.

Issues - 6 Definition of Integration rules Description of how components can[13] Comparison of properties/tradeoff analysis[12] We want to be able to have a set of rules that can be used to help COTS developers in their integration efforts. The definition of integration rules can affect how components can be combined - a catalog - rules that evolve from a synthesis of the activities done to mitigate mismatches These integration rules are specific to attribute values (i.e. attributes of a component that are of interest to integration)  They include lists of circumstances, which are hard or easy - includes and classifies known mitigators. It can serve as a knowledge base of integration rules.

7. How to perform trade-off analysis of system properties in CBS Importance M L aw rs L M H rr nm jk Difficulty

8. Predict system properties from component properties H Importance M L aw L M H rr nm rs jk Difficulty

9. Component patterns and architectural styles in CBS Importance M L aw L M H rr nm rs jk Difficulty

11. Architectural Pattern and CBS Importance M L aw L M H rr nm rs jk Difficulty